How to Implement AI Noise Reduction in WebRTC

Introduction

A. Overview of WebRTC and noise issues

WebRTC (Web Real-Time Communication) has revolutionized browser-based communication, enabling direct peer-to-peer audio, video, and data transfer without the need for plugins or additional software. However, as with any real-time communication technology, WebRTC faces challenges related to audio quality, particularly when it comes to background noise. This noise can significantly impact the clarity and effectiveness of communication, leading to misunderstandings and frustration for users.

B. Importance of AI noise reduction in WebRTC

The implementation of AI noise reduction in WebRTC is crucial for several reasons:

1. Enhanced user experience: By reducing background noise, AI algorithms can dramatically improve the clarity of audio, leading to more natural and enjoyable conversations.
2. Increased productivity: In professional settings, clearer audio means fewer misunderstandings and less time spent repeating information, ultimately boosting productivity.
3. Accessibility: For users with hearing impairments or those in noisy environments, AI noise reduction can make the difference between being able to participate in a conversation or not.
4. Competitive advantage: As users become more discerning about audio quality, platforms that offer superior noise reduction capabilities are likely to gain a competitive edge.

C. Brief explanation of AI noise reduction principles

AI noise reduction in WebRTC leverages machine learning algorithms to distinguish between desired speech and unwanted background noise. These algorithms are trained on vast datasets of speech and noise samples, allowing them to recognize and separate speech patterns from various types of background noise in real-time. The AI system then suppresses or removes the identified noise while preserving the quality and naturalness of the speech signal.

AI Noise Reduction: Technical Principles

A. Machine learning algorithms for noise detection

AI noise reduction employs sophisticated machine learning algorithms to detect and classify various types of noise:

1. Supervised learning: Algorithms are trained on labeled datasets of clean speech and noise, learning to distinguish between the two.
2. Unsupervised learning: These methods can adapt to new noise environments without prior training, identifying patterns that distinguish speech from noise.
3. Deep learning: Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) can be used to analyze the spectral and temporal characteristics of audio signals, identifying noise components with high accuracy.

B. Neural networks for speech enhancement

Once noise is detected, neural networks are employed to enhance the speech signal:

1. Denoising autoencoders: These neural networks learn to reconstruct clean speech from noisy input.
2. Generative adversarial networks (GANs): GANs can be used to generate clean speech, with a discriminator network ensuring the output closely matches natural speech.
3. Time-domain audio separation networks: These directly process raw audio waveforms, separating speech from noise in the time domain.

C. Real-time processing considerations

Implementing AI noise reduction in WebRTC requires careful consideration of real-time processing constraints:

1. Latency: The noise reduction algorithm must operate with minimal delay to maintain the real-time nature of the communication.
2. Computational efficiency: The algorithm needs to be optimized to run efficiently on a wide range of devices, from smartphones to desktop computers.
3. Adaptive processing: The system should be able to adapt to changing noise conditions in real-time without introducing artifacts or disruptions.
4. Integration with existing WebRTC audio processing pipeline: The AI noise reduction should work seamlessly with other audio processing features of WebRTC, such as echo cancellation and automatic gain control.

By leveraging these AI techniques, WebRTC can achieve significantly improved noise reduction compared to traditional methods, leading to clearer, more natural communication even in challenging acoustic environments.

Implementing AI Noise Reduction with Tencent RTC

Explore online demo

Tencent RTC offers a powerful solution for implementing AI noise reduction in WebRTC applications. Our AI Noise Suppression feature, developed by Tencent's Tianlai Labs, provides state-of-the-art noise elimination capabilities. 

You can also enter our Experience Center to try the excellent sound effects brought by AI Noise Suppression online.

Activate AI Noise Suppression

Tencent RTC Conference now comes with AI Noise Suppression functionality enabled by default. Users can enjoy high-quality noise suppression within their applications without the need for additional configurations or actions.

We also provide the RTCAIDenoiser plugin, which can be used in conjunction with the TRTC Web SDK to reduce noise during calls and minimize the impact of environmental sounds on communication. In the following section, we will introduce how to use the RTCAIDenoiser plugin when developing TRTC applications

Prerequisites

From 1 April 2023, TRTC monthly subscription Premium and higher is required to use the AI noise reduction function.

Supported browsers: Chrome 66+, Edge 79+, Safari 14.1+, Firefox 76+.

For better use of AI noise reduction, it is recommended that you use the latest version of Chrome.

Notes:

If there is background music captured by your microphone, RTCAIDenoiser may eliminate it as noise.

Feature Description

Step1. Install RTCAIDenoiser

npm install rtc-ai-denoiser@latest

The RTCAIDenoiser plugin needs to be installed in the same scope as TRTC.

import TRTC from 'trtc-js-sdk';
import RTCAIDenoiser from 'rtc-ai-denoiser';

Step2. Integrated RTCAIDenoiser

Dynamically loading file dependencies: The RTCAIDenoiser plugin relies on a number of files. To ensure that your browser can load and run these files properly, you need to complete the following steps.

Publish the denoiser-wasm.js file from the node_modules/rtc-ai-denoiser/assets directory to a CDN or static resource server and under the same public path. When creating RTCAIDenoiser instances later, you need to pass in the URL of the above public path and the plugin will load the dependency files dynamically.

If the Host URL of the file in the assets directory does not match the Host URL of the web application, you need to enable the CORS policy for accessing the file domain.
You cannot place assets directory files under an HTTP service, as loading HTTP resources under an HTTPS domain is prohibited by browser security policies.

Step3. Init RTCAIDenoiser

1.Reference Quick Start Call to implement a basic audio/video call process.

2.init RTCAIDenoiser

// Create an instance, passing in the public path where the files in the assets directory are located
const rtcAIDenoiser = new RTCAIDenoiser({ assetsPath: './assets' });

3.create denoiserProcessor instance

const processor = await rtcAIDenoiser.createProcessor({
  sdkAppId,
  userId,
  userSig
});

4.handle localStreams that need to be published.

// init stream
const localStream = TRTC.createStream({ video: true, audio: true });
await localStream.initialize();

// adding noise suppression to localStream
await processor.process(localStream);
// publish
await client.publish(localStream);

5.Control whether the plugin is turned on or off: call the enable method and the disable method.

if (processor.enabled) {
  await processor.disable();
} else {
  await processor.enable();
}

6.Dump the audio data during the noise suppression process: call the startDump method to start and the stopDump method to end, and listen to the ondumpend callback to get the audio and video data.

processor.on('ondumpend', ({ blob, name }) => {
  const url = window.URL.createObjectURL(blob);

  let anchor = document.createElement('a');
  anchor.href = url;
  anchor.download = `${name}-${Date.now()}.wav`;
  anchor.click();
  window.URL.revokeObjectURL(url);
  anchor.href = '';
});

Advantages of Tencent RTC's Solution

1. Advanced AI Algorithms: Leverages cutting-edge AI technology from Tianlai Labs for superior noise detection and suppression.
2. Versatility: Effective across a wide range of environments and noise types.
3. Integration with TUIRoomKit: Seamless integration with Tencent's UI components for quick implementation.
4. Real-time Processing: Designed for low-latency applications, ensuring no noticeable delay in communication.