Starter Deal! First 3 month from only  $9.9 /month!
Starter Deal! First 3 month from only  $9.9 /month!
Grab It Now 
RTC Engine
Overview
  • Web
    • Run Sample Demo
    • SDK Quickstart
    • Basic Features
      • Screen Sharing
      • Live Streaming
      • Media Device
      • Audio Volume
      • Set Encoding Profile
      • Detect Network Quality
      • Detect Capabilities
    • Advance Features
      • Enable AI Denoiser
      • Enable Audio Mixer
      • Enable Watermark
      • Enable Beauty and Effects
      • Data Messages
      • Custom Capturing and Rendering
    • API List
    • Released Notes
    • Supported Platforms
    • Web FAQs
      • Optimize Multi-Person Video Calls
      • Handle Autoplay Restriction
      • Handle Firewall Restriction
      • Others
  • Android
    • Integration
      • 1.API Examples
      • 2.Importing the SDK
      • 3.Entering a Room
      • 4.Subscribing to Audio/Video Streams
      • 5.Publish Audio/Video Streams
      • 6.Exiting a Room
      • 7.Sensing Network Quality
      • 8.Enabling Screen Sharing
      • 9.Setting Video Quality
      • 10.Rotating Videos
    • Testing Newwork Quality
    • Custom Capturing and Rendering
    • Custom Audio Capturing and Playback
    • Client APIs
      • Overview
      • TRTCCloud
      • TRTCStatistics
      • TRTCCloudListener
      • TXAudioEffectManager
      • TXBeautyManager
      • TXDeviceManager
      • Type Definition
      • Deprecated Interface
      • Error Codes
    • Solution
      • Real-Time Chorus (TUIKaraoke)
        • Quick Integration
        • Implementation Steps
        • Song Synchronization
        • Lyric Synchronization
        • Vocal Synchronization
        • Mixing Stream Solution
        • TRTCKaraoke APIs
        • FAQs
    • Release Notes
  • iOS
    • Integration
      • 1.API Examples
      • 2.Importing the SDK
      • 3.Entering a Room
      • 4.Subscribing to Audio/Video Streams
      • 5.Publish Audio/Video Streams
      • 6.Exiting a Room
      • 7.Sensing Network Quality
      • 8.Enabling Screen Sharing
      • 9.Setting Video Quality
      • 10.Rotating Videos
    • Testing Network Quality
    • Custom Capturing and Rendering
    • Custom Audio Capturing and Playback
    • Client APIs
      • Overview
      • TRTCCloud
      • TRTCCloudDelegate
      • TRTCStatistics
      • TXAudioEffectManager
      • TXBeautyManager
      • TXDeviceManager
      • Type Definition
      • Deprecated Interface
      • ErrorCode
    • Solution
      • Quick Integration
      • Implementation Steps
      • Song Synchronization
      • Lyric Synchronization
      • Vocal Synchronization
      • Mixing Stream Solution
      • TRTCKaraoke APIs
      • FAQs
    • Release Notes
  • macOS
    • Integration
      • 1.API Examples
      • 2.Importing the SDK
      • 3.Entering a Room
      • 4.Subscribing to Audio/Video Streams
      • 5.Publish Audio/Video Streams
      • 6.Exiting a Room
      • 7.Sensing Network Quality
      • 8.Enabling Screen Sharing
      • 9.Sharing Computer Audio
      • 10.Setting Video Quality
      • 11.Rotating Videos
    • Testing Hardware Devices
    • Testing Network Quality
    • Custom Capturing and Rendering
    • Custom Audio Capturing and Playback
    • Client APIs
      • Overview
      • TRTCCloud
      • TRTCCloudDelegate
      • TRTCStatistics
      • TXAudioEffectManager
      • TXBeautyManager
      • TXDeviceManager
      • Type Definition
      • Deprecated Interface
      • ErrorCode
      • Release Notes
    • Release Notes
  • Windows
    • Integration
      • 1.API Examples
      • 2.Importing the SDK
      • 3.Entering a Room
      • 4.Subscribing to Audio/Video Streams
      • 5.Publish Audio/Video Streams
      • 6.Exiting a Room
      • 7.Sensing Network Quality
      • 8.Enabling Screen Sharing
      • 9.Setting Video Quality
      • 10.Rotating Videos
    • Testing Hardware Devices
    • Testing Network Quality
    • Custom Capturing and Rendering
    • Custom Audio Capturing and Playback
    • Client APIs
      • Overview
      • ITRTCCloud
      • ITRTCStatistics
      • TRTCCloudCallback
      • ITXAudioEffectManager
      • ITXDeviceManager
      • Type Definition
      • Deprecated Interface
      • Error Codes
    • Release Notes
  • Electron
    • Integration
      • 1.API Examples
      • 2.Importing the SDK
      • 3.Entering a Room
      • 4.Subscribing to Audio/Video Streams
      • 5.Publish Audio/Video Streams
      • 6.Exiting a Room
      • 7.Sensing Network Quality
      • 8.Enabling Screen Sharing
      • 9.Sharing Computer Audio
      • 10.Setting Video Quality
      • 11.Rotating Videos
    • Client APIs
      • Overview
      • Error Codes
  • Flutter
    • Integration
      • 1.API Examples
      • 2.Importing the SDK
      • 3.Entering a Room
      • 4.Subscribing to Audio/Video Streams
      • 5.Publish Audio/Video Streams
      • 6.Exiting a Room
      • 7.Sensing Network Quality
      • 8.Enabling Screen Sharing
      • 9.Sharing Computer Audio
      • 10.Setting Video Quality
      • 11.Rotating Videos
    • Client APIs
      • Overview
      • Error Codes
  • Unity
    • Integration
      • 1.API Examples
      • 2Importing the SDK
    • Client APIs
      • Overview
      • Error Codes
  • Qt
    • Integration
      • 1.Importing the SDK
  • Overview
    • Overview
  • Concepts
  • Features
  • Performance Statistics
  • Pricing
    • RTC-Engine Packages
    • Billing of On-Cloud Recording
    • Billing of MixTranscoding and Relay to CDN
    • Billing Explanation for Subscription Package Duration
    • Billing of Monitoring Dashboard
    • Free Minutes
    • Pay-As-You-Go
  • Tencent RTC Quickplay: Experience Ultimate Real-Time Audio and Video Interaction!
  • FAQs
    • FAQs for Beginners
    • Migration Guide
      • Twilio Video to Tencent RTC
      • Billing
      • Features
      • UserSig
      • Firewall Restrictions
      • How to Downsize Installation Package
      • TRTCCalling for Web
      • Audio and Video Quality
      • Others
RTC Engine

Custom Audio Capturing and Playback

This document describes how to use the TRTC SDK to implement custom audio capturing and rendering.

Custom Audio Capturing

The custom audio capturing feature of the TRTC SDK can be used in two steps: enabling the feature and sending audio frames to the SDK. For detailed directions of specific APIs, see below. We also provide API examples for different platforms:

Enabling custom audio capturing

To enable the custom audio capturing feature of the TRTC SDK, you need to call the enableCustomAudioCapture API of TRTCCloud. Below is the sample code:
Android
iOS&Mac
Windows
TRTCCloud mTRTCCloud = TRTCCloud.shareInstance();
mTRTCCloud.enableCustomAudioCapture(true);

self.trtcCloud = [TRTCCloud sharedInstance];
[self.trtcCloud enableCustomAudioCapture:YES];

liteav::ITRTCCloud* trtc_cloud = liteav::ITRTCCloud::getTRTCShareInstance();
trtc_cloud->enableCustomAudioCapture(true);


Sending custom audio frames

You can use the sendCustomAudioData API of TRTCCloud to populate the TRTC SDK with your own audio data. Below is the sample code:
Android
iOS&Mac
Windows
TRTCCloudDef.TRTCAudioFrame trtcAudioFrame = new TRTCCloudDef.TRTCAudioFrame();
trtcAudioFrame.data = data;
trtcAudioFrame.sampleRate = sampleRate;
trtcAudioFrame.channel = channel;
trtcAudioFrame.timestamp = timestamp;
mTRTCCloud.sendCustomAudioData(trtcAudioFrame);

TRTCAudioFrame *audioFrame = [[TRTCAudioFrame alloc] init];
audioFrame.channels = audioChannels;
audioFrame.sampleRate = audioSampleRate;
audioFrame.data = pcmData;

[self.trtcCloud sendCustomAudioData:audioFrame];

liteav::TRTCAudioFrame frame;
frame.audioFormat = liteav::TRTCAudioFrameFormatPCM;
frame.length = buffer_size;
frame.data = array.data();
frame.sampleRate = 48000;
frame.channel = 1;
getTRTCShareInstance()->sendCustomAudioData(&frame);

notice
Using sendCustomAudioData may cause AEC to fail.

Getting Raw Audio Data

The audio module is a highly complex module, and the TRTC SDK needs to strictly control the capturing and playback logic of audio devices. In some cases, to get the audio data of a remote user or audio captured by the local mic, you can use the APIs of TRTCCloud for different platforms. We also provide API examples for those platforms:

Setting audio callback

Android
iOS&Mac
Windows
mTRTCCloud.setAudioFrameListener(new TRTCCloudListener.TRTCAudioFrameListener() {
@Override
public void onCapturedRawAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {

}

@Override
public void onLocalProcessedAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {

}

@Override
public void onRemoteUserAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame, String s) {

}

@Override
public void onMixedPlayAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {

}

@Override
public void onMixedAllAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {
// For more information, see the custom rendering tool class `com.tencent.trtc.mediashare.helper.CustomFrameRender` in `TRTC-API-Example`
}
});

[self.trtcCloud setAudioFrameDelegate:self];
// MARK: - TRTCAudioFrameDelegate
- (void)onCapturedRawAudioFrame:(TRTCAudioFrame *)frame {
NSLog(@"onCapturedRawAudioFrame");
}

- (void)onLocalProcessedAudioFrame:(TRTCAudioFrame *)frame {
NSLog(@"onLocalProcessedAudioFrame");
}

- (void)onRemoteUserAudioFrame:(TRTCAudioFrame *)frame userId:(NSString *)userId {
NSLog(@"onRemoteUserAudioFrame");
}

- (void)onMixedPlayAudioFrame:(TRTCAudioFrame *)frame {
NSLog(@"onMixedPlayAudioFrame");
}

- (void)onMixedAllAudioFrame:(TRTCAudioFrame *)frame {
NSLog(@"onMixedAllAudioFrame");
}

// Set custom audio data callback
liteav::ITRTCCloud* trtc_cloud = liteav::ITRTCCloud::getTRTCShareInstance();
trtc_cloud->setAudioFrameCallback(callback)

// Callback APIs for custom audio

virtual void onCapturedRawAudioFrame(TRTCAudioFrame* frame) {
}

virtual void onLocalProcessedAudioFrame(TRTCAudioFrame* frame) {
}

virtual void onPlayAudioFrame(TRTCAudioFrame* frame, const char* userId) {
}

virtual void onMixedPlayAudioFrame(TRTCAudioFrame* frame) {
}

notice
Do not perform time-consuming operations with any of the above callback functions. We recommend that you copy the data to another thread for processing to avoid AEC failure and choppy audio.
The data called back by the above callback functions can only be read and copied. Modifying the data may lead to unexpected results.