Audio Connector
The Audio Connector captures call audio and metadata in real time (while the call is in progress) either directly from cloud platforms such as Amazon Connect, Genesys, and Five9, or through the Collectors via NATS (see Audio Connector and NATS below). The Audio Connector processes the call's audio and metadata and streams the data using Kafka to the Conversation Service (metadata), the ASR (audio for transcription), and the Media Handler (audio for storage).
Note
The relevant integration topics cover information on how calls are captured from specific platforms (click here). For system deployment details specific to your environment, please refer to your Solution Design Document.
Once the Audio Connector knows a conversation has started, it requests a Conversation ID from the Conversation Service over GRPC. This universal ID is assigned to the conversation, where it is used to identify the conversation throughout the system, which allows the system to query specific conversations and tie conversations together. The Audio Connector modernizes the metadata, mapping it to metadata fields that the Conversation Service is expecting. The Conversation Service can then seamlessly store, query, and filter captured calls by using this uniform metadata. The Audio Connector converts the audio into a "normal form", PCM 16 Stereo, where in most scenarios the agent is on one channel and the customer is on the other channel - having all audio in PCM 16 stereo allows it to be easily consumable and stored in a consistent manner by the Media Handler. Real-time audio is streamed to the Media Handler over GRPC for Live Monitor (go to Conversations > Live Monitor to view live calls).
The Audio Connector also works with the Policy Manager to determine if a conversation should be recorded. The Audio Connector receives a list of policies that filter by the conversation's metadata and checks if a Blocklist policy is applicable. If the call matches a Blocklist policy, then the call is not recorded. The Conversation Service also checks with the Policy Manager, and if a conversation shouldn't be recorded, it labels the conversation as not recorded due to a Blocklist policy.
Audio Connector and NATS
As mentioned above, for integrations using Collectors the Audio Connector captures conversation audio and metadata in real time via NATS.
The Audio Connector NATS Consumer receives a single stream from each NATS partition containing conversation metadata and media chunks for all conversations in real time. These streamed conversations are added to the NATS Consumer processing queue. As the NATS Consumer processes conversations it splits the stream into individual conversations, linking media chunks/metadata (including events). The Audio Connector then sends the raw data for each conversation to Kafka, and sends the media to the Media Handler and the metadata to the Conversation Service via GRPC.
If the system detects an issue with the NATS Consumer, then the NATS Consumer stops consuming the NATS stream and finishes processing the conversation queue, and then restarts. Once restarted, the NATS consumer begins consuming the NATS stream again and continues to process conversations.
The system detects any inconsistencies in conversation processing using the reconciliation services. For each conversation, a hash is generated by the Collector using the conversation data. Once the conversation is processed, the reconciliation service also generates a hash and the two hashes are compared. If there are inconsistencies in the hashes then the system knows that there was an issue processing the conversation. When an issue is detected, the Audio Connector requests a specific subject from NATS containing the media and metadata for the specific conversation (not containing call events to avoid triggering event based processes throughout the system).