Conversational AI Server Callback

This document introduces the events generated by the cloud APIs related to AI services (Conversational AI), which are notified to your server via HTTP/HTTPS requests. You can provide the relevant configuration information to Tencent Cloud to enable this service. You can also use it in conjunction with Tencent RTC's Event Callbacks to implement more custom logic.

Configuration Information

Tencent RTC Console supports self-configured callback information. Once configured, you can receive event callback notifications. For detailed instructions, please see Callback Configuration.
Note:
You need to prepare the following information in advance:
Required: the HTTP/HTTPS server address to receive callback notifications.
Optional: The key for calculating the signature, which is up to 32 characters defined by you, consisting of uppercase and lowercase letters and numbers.

Timeout Retry

If the Event Callback Server does not receive a response from your server within 5 seconds after sending a Message Notification, it is considered a Notification Failure. After the first failure, an immediate retry is attempted. Subsequent retries will occur at 10-second intervals until the Message Retention Time exceeds 1 minute, after which no further retries will be made.

Format of the Event Callback Message

Event callback messages are dispatched to your server via HTTP/HTTPS POST requests, in which:
Character Encoding Format: UTF-8.
Request: The body is in JSON format.
Response: HTTP STATUS CODE = 200. The server ignores the specific content of the response package. For the sake of a friendly protocol, it is recommended that the customer carry the response content JSON: {"code":0}.
Package Example: The following is a package example for the "Successful Initiation of AI Conversation Task" event.
{
"EventGroupId": 9,
"EventType": 901,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "hKPD2Q7kBVzu-6ezFiqmcEBJQCykqbZrS9OOTE46uYlb4NvQDIaEXlpOlLXFtGBiado5oP0zfLDZs",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"Status": 0
}
}
}

Parameter Description

Callback Message Parameters

The header of the event callback message contains the following fields:
Field name.
Value.
Content-Type
application/json
Sign
Signature Value
SdkAppId
sdk application id
The body of the event callback message includes the following fields:
Field name.
Type
Meaning
EventGroupId
Number
Event Group ID, fixed at 4 for mix streaming and relay events
EventType
Number
Type of event for callback notification
CallbackMsTs
Number
The Unix timestamp (in milliseconds) when the event callback server sends a callback request to your server
EventInfo
JSON Object

Event Group ID

Field name.
Value.
Meaning
EVENT_GROUP_AI_SERVICE
9
AI Service Event Group

Event Type

Field name.
Value.
Meaning
EVENT_TYPE_AI_SERVICE_START
901
AI Task Start Status Callback
EVENT_TYPE_AI_SERVICE_STOP
902
AI Task End Status Callback
EVENT_TYPE_AI_SERVICE_MSG
903
Callback the complete sentence recognized by ASR or the complete content returned by LLM
EVENT_TYPE_AI_START_OF_SPEECH
904
Callback for the start of a sentence recognized by ASR
EVENT_TYPE_AI_SPEAKING_FINISHED
905
Callback when AI finishes speaking in a conversation round
EVENT_TYPE_AI_METRIC_MESSAGE
906
Metrics callback for invoking LLM/TTS in AI services
EVENT_TYPE_AI_ERROR_METRIC_CALLBACK
908
Error callback for invocation metrics in AI services
EVENT_TYPE_AI_SESSION_STATUS_CALLBACK
909
Callback for AI session readiness, indicating that the audio and video channels are established and ready for conversation.

Definition of Event Information When Event Type Is 901:

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload.Status
Number
0: AI Task Started Successfully
1: AI Task Failed to Start
{
"EventGroupId": 9,
"EventType": 901,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"Status": 0
}
}
}

Definition of Event Information When Event Type Is 902:

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload.LeaveCode
Integer
0: Task exited after normal call to stop interface
1: Task exited after the business removed the transcription bot
2: Task exited after the business dissolved the room
3: Tencent RTC server removed the bot
4: Tencent RTC server dissolved the room
98: Internal exception error. It is recommended that the business retries
99: Represents no user stream in the room except the transcription bot. Exits after exceeding the specified time
{
"EventGroupId": 9,
"EventType": 902,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"LeaveCode": 0
}
}
}

Definition of Event Information When Event Type Is 903:

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload
JSON Object
It is a JSON object, consistent with the client custom message callback format
{
"UserId":"",
"Text":"",
"StartTimeMs":1234,
"EndTimeMs":1269,
"RoundId":"xxxxxx" // A unique ID for a single conversation round
}
{
"EventGroupId": 9,
"EventType": 903,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"UserId":"",
"Text":"",
"StartTimeMs":1234,
"EndTimeMs":1269,
"RoundId":"xxxxxx"
}
}
}

Definition of Event Information When Event Type Is 904:

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload
JSON Object
for JSON object
{
"UserId": "xxx",
"RoundId": "xxxxx" // A unique ID for a single conversation round
}
{
"EventGroupId": 9,
"EventType": 904,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"UserId": "xxx",
"RoundId": "xxxxx"
}
}
}

Definition of Event Information When Event Type Is 905:

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload
JSON Object
for JSON object
{
"UserId": "UserId", // The UserId of the AI chatbot
"RoundId": "RoundId", // The RoundId of the current conversation
"Text": "Text"
}
{
"EventGroupId": 9,
"EventType": 905,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"UserId": "UserId",
"RoundId": "RoundId",
"Text": "Text"
}
}
}

Definition of Event Information When Event Type Is 906:

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload
JSON Object
for JSON object
{
"Metric": "llm_network_latency", // Metric name
"Value": 218, // Metric
"Tag": {
"RoundId": "070c4908-105", // Conversation round ID
}
}

The metric names to be called are as follows:
asr_latency
llm_network_latency
llm_first_token
tts_network_latency
tts_first_frame_latency
tts_discontinuity
interruption
{
"EventGroupId": 9,
"EventType": 906,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"Metric": "llm_first_token",
"Value": 218,
"Tag": {
"RoundId": "070c4908-1057-4ced-a949-356bf11848bc"
}
}
}
}

Definition of Event Information When Event Type Is 908:

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload
JSON Object
for JSON object
{
"Metric": "llm_error", // Metric name
"Tag": {
"RoundId": "070c4908-1057-4ced-a949-356bf11848bc",
"Code": 0, // Service error code
"Message": "" // Detailed description of the error message
}
}

Error metric names are as follows:
asr_latency
llm_network_latency
llm_first_token
tts_network_latency
tts_first_frame_latency
tts_discontinuity
interruption
{
"EventGroupId": 9,
"EventType": 908,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"Metric": "llm_error",
"Tag": {
"RoundId": "070c4908-1057-4ced-a949-356bf11848bc",
"Code": 0,
"Message": ""
}
}
}
}

Definition of Event Information When Event Type Is (909):

Field name.
Type
Meaning
EventMsTs
String
The Unix timestamp of when the event occurred, in milliseconds
TaskId
String
AI Task ID
RoomId
String
Tencent RTC Room ID
RoomIdType
Integer
0: Indicates Numeric Room Number
1: Indicates String Room Number
Payload
JSON Object
for JSON object
{
"Status": "session_ready" // Indicates that the audio and video channel is established and ready for conversation.
}
{
"EventGroupId": 9,
"EventType": 909,
"CallbackTs": 1687770730166,
"EventInfo": {
"EventMsTs": 1622186275757,
"TaskId": "xx",
"RoomId": "1234",
"RoomIdType": 0,
"Payload": {
"Status": "session_ready"
}
}
}

Calculating signature

Signatures are calculated using the HMAC SHA256 encryption algorithm. After your event callback server receives the callback message, it calculates the signature in the same manner. If they match, it confirms that it is an event callback from Tencent's Real-Time Communication, and not a falsification. The signature calculation is as illustrated below:
// In the signature 'Sign' calculation formula, the 'key' is the encryption key used for calculating the signature 'Sign'.
Sign = base64(hmacsha256(key, body))
Note:
Body refers to the original package body of the callback request received by you, avoid any transformations, as illustrated below:
body="{\n\t\"Ebody="{\"EventGroupId\":7,\"EventType\":701,\"CallbackMsTs\":1701937900012,\"EventInfo\":{\"EventMsTs\":1701937900012,\"TaskId\":\"WMdqEeEgj2ksqnyUsuXC+qLkVypGmwjrgh1JC6ZefVP+rvsidDnZsAw8uWgX0XRGvdSVfAMunise2kcZaefdgHvx3-M2v6fmTjRNgg..\",\"Status\":0}}"ventGroupId\":\t1,\n\t\"EventType\":\t103,\n\t\"CallbackTs\":\t1615554923704,\n\t\"EventInfo\":\t{\n\t\t\"RoomId\":\t12345,\n\t\t\"EventTs\":\t1608441737,\n\t\t\"UserId\":\t\"test\",\n\t\t\"UniqueId\":\t1615554922656,\n\t\t\"Role\":\t20,\n\t\t\"Reason\":\t1\n\t}\n}"

Verify Signature Example

Java
Python
PHP
Golang
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.util.Base64;
//# Function: Third-party callback sign verification
//# Parameters:
//# key: Key configured in the console
//# body: Tencent Cloud callback returned body
//# sign: Tencent Cloud callback returned signature value
//# Returned values:
//# Status: OK indicates validation passed, FAIL indicates validation failed, refer to Info for details
//# Info: Success/Failure Information

public class checkSign {
public static String getResultSign(String key, String body) throws Exception {
Mac hmacSha256 = Mac.getInstance("HmacSHA256");
SecretKeySpec secret_key = new SecretKeySpec(key.getBytes(), "HmacSHA256");
hmacSha256.init(secret_key);
return Base64.getEncoder().encodeToString(hmacSha256.doFinal(body.getBytes()));
}
public static void main(String[] args) throws Exception {
String key = "123654";
String body = "{\n" + "\t\"EventGroupId\":\t2,\n" + "\t\"EventType\":\t204,\n" + "\t\"CallbackTs\":\t1664209748188,\n" + "\t\"EventInfo\":\t{\n" + "\t\t\"RoomId\":\t8489,\n" + "\t\t\"EventTs\":\t1664209748,\n" + "\t\t\"EventMsTs\":\t1664209748180,\n" + "\t\t\"UserId\":\t\"user_85034614\",\n" + "\t\t\"Reason\":\t0\n" + "\t}\n" + "}";
String Sign = "kkoFeO3Oh2ZHnjtg8tEAQhtXK16/KI05W3BQff8IvGA=";
String resultSign = getResultSign(key, body);

if (resultSign.equals(Sign)) {
System.out.println("{'Status': 'OK', 'Info': 'Verification passed'}");
} else {
System.out.println("{'Status': 'FAIL', 'Info': 'Verification Failed'}");
}
}
}
# -*- coding: utf8 -*-
import hmac
import base64
from hashlib import sha256

# Function: Third-party callback sign verification
# The parameters are as follows:
# key: Console-configured Key
# body: Tencent Cloud callback returned body
# sign: Tencent Cloud callback returned signature value
# Returned Values:
# Status: OK indicates validation passed, FAIL indicates validation failed, refer to Info for details
# Info: Success/Failure Information

def checkSign(key, body, sign):
temp_dict = {}
computSign = base64.b64encode(hmac.new(key.encode('utf-8'), body.encode('utf-8'), digestmod=sha256).digest()).decode('utf-8')
print(computSign)
if computSign == sign:
temp_dict['Status'] = 'OK'
temp_dict['Info'] = 'Verification passed'
return temp_dict
else:
temp_dict['Status'] = 'FAIL'
temp_dict['Info'] = 'Verification Failed'
return temp_dict

if __name__ == '__main__':
key = '123654'
body = "{\n" + "\t\"EventGroupId\":\t2,\n" + "\t\"EventType\":\t204,\n" + "\t\"CallbackTs\":\t1664209748188,\n" + "\t\"EventInfo\":\t{\n" + "\t\t\"RoomId\":\t8489,\n" + "\t\t\"EventTs\":\t1664209748,\n" + "\t\t\"EventMsTs\":\t1664209748180,\n" + "\t\t\"UserId\":\t\"user_85034614\",\n" + "\t\t\"Reason\":\t0\n" + "\t}\n" + "}"
sign = 'kkoFeO3Oh2ZHnjtg8tEAQhtXK16/KI05W3BQff8IvGA='
result = checkSign(key, body, sign)
print(result)
<?php

class TlsEventSig {
private $key = false;
private $body = false;
public function __construct( $key, $body ) {
$this->key = $key;
$this->body = $body;
}

private function __hmacsha256() {
$hash = hash_hmac( 'sha256', $this->body, $this->key, true );
return base64_encode( $hash);
}
public function genEventSig() {
return $this->__hmacsha256();
}
}

$key="789";
$data="{\n\t\"EventGroupId\":\t1,\n\t\"EventType\":\t101,\n\t\"CallbackTs\":\t1608086882372,\n\t\"EventInfo\":\t{\n\t\t\"RoomId\":\t20222,\n\t\t\"EventTs\":\t1608086882,\n\t\t\"UserId\":\t\"222222_phone\"\n\t}\n}";

$api = new TlsEventSig($key, $data);
echo $api->genEventSig();
package main
import "fmt"
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
)

func main () {
var data = "{\n\t\"EventGroupId\":\t1,\n\t\"EventType\":\t101,\n\t\"CallbackTs\":\t1608086882372,\n\t\"EventInfo\":\t{\n\t\t\"RoomId\":\t20222,\n\t\t\"EventTs\":\t1608086882,\n\t\t\"UserId\":\t\"222222_phone\"\n\t}\n}"
var key = "789"

//JSRUN Engine 2.0, supporting online execution in up to 30 languages, complete emulation of online interactive input/output.
fmt.Println(hmacsha256(data,key))
}

func hmacsha256(data string, key string) string {
h := hmac.New(sha256.New, []byte(key))
h.Write([]byte(data))
return base64.StdEncoding.EncodeToString(h.Sum(nil))
}