TUIMediaMixingManager

Introduction

TUIMediaMixingManager is a local multimedia mixing manager that supports mixing multiple local video streams into one video stream for streaming.
Installation
// npm
npm i @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3 --save

// pnpm
pnpm i @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3 --save

// yarn
yarn add @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3

Example code:
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();

API

Method
Description
Set the program path for the local mixing service process
Set the local mixing video preview window and area
Add a multimedia source
Remove a multimedia source
Update a multimedia source
Set camera capture parameters
Start local mixing and streaming
Stop local mixing and streaming
Modify local mixing parameters

Type Definitions

Type
Description
Camera capture parameters
Local mixing configuration parameters
Media source data
TUIRect
Rectangular coordinate area, including left, top, right, bottom properties
Screen/window data
Mixed video encoding parameters

Enumerations

Enum
Description
Camera capture mode
Media source fill mode
Media source mirror type
Media source rotation angle
Media source type
Resolution mode (landscape resolution/vertical resolution)
Video resolution

API explaination

setMediaServerPath()

Set the program path for the local mixing service process. After creating an instance object, this method must be called first for other methods to take effect.
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();

async function initMediaServer() {
const isPackaged = location.href.startsWith("file"); // Same with Electron `app.isPackaged`
const resourcesPath = (globalThis as any).process.resourcesPath;
let mediaServerPath = '';
if (isPackaged) {
mediaServerPath = `${resourcesPath}\\liteav_media_server.exe`;
} else {
const appPath = ''; // Value from Electron main process: app.getAppPath()
mediaServerPath = `${appPath}\\node_modules\\trtc-electron-sdk\\build\\Release\\liteav_media_server.exe`;
}
mediaMixingManager.setMediaServerPath(mediaServerPath);
}

initMediaServer();
Parameters
Parameter
Type
Validation
Default value
Description
path
string
-
-
Program path for the local mixing service process
Return Promise<void>

setDisplayParams()

Set the local mixing video preview window and area
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();

// Value from Electron main BrowserWindow.getNativeWindowHandle()
const nativeWindowHandle = new Uint8Array(8);

const previewDOMNode = document.getElementByID("preview-media-mixing");
const clientRect = previewDOMNode.getBoundingClientRect();
mediaMixingManager.setDisplayParams(window.nativeWindowHandle, {
left: clientRect.left * window.devicePixelRatio, // Notice: window.devicePixelRatio
right: clientRect.right * window.devicePixelRatio, // Notice: window.devicePixelRatio
top: clientRect.top * window.devicePixelRatio, // Notice: window.devicePixelRatio
bottom: clientRect.bottom * window.devicePixelRatio, // Notice: window.devicePixelRatio
});
Parameters
Parameter
Type
Validation
Default value
Description
windowID
Uint8Array
-
-
Mixed video preview window ID, obtained through Electron API BrowserView.getNativeWindowHandle()
rect
TUIRect
-
-
Mixed video preview area position and size
Return void

addMediaSource()

Add media source
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
const mediaSource:TUIMediaSource = {
sourceType: TUIMediaSourceType.kCamera,
sourceId: '', // camera ID, screen ID, window ID or image path
zOrder: 1, // should be unique among all media sources
rect: {
left: 0,
top: 0,
right: 640,
bottom: 360
},
};
mediaMixingManager.addMediaSource(mediaSource);
Parameters
Parameter
Type
Validation
Default value
Description
mediaSource
-
-
Media source information
Return Promise<TUIRect> Default position in mixing video image

removeMediaSource()

Remove media source
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
const mediaSource:TUIMediaSource = {
sourceType: TUIMediaSourceType.kCamera, // required
sourceId: '', // required
zOrder: 1, // any valid value
rect: { // any valid value
left: 0,
top: 0,
right: 640,
bottom: 360
},
};
mediaMixingManager.removeMediaSource(mediaSource);
Parameters
Parameter
Type
Validation
Default value
Description
mediaSource
-
-
Media source information
Return Promise<void>

updateMediaSource()

Update media source
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
const mediaSource:TUIMediaSource = {
sourceType: TUIMediaSourceType.kCamera, // required
sourceId: '', // required
zOrder: 1, // should be unique among all media sources
rect: { // any valid value
left: 0,
top: 0,
right: 640,
bottom: 360
},
};
mediaMixingManager.updateMediaSource(mediaSource);
Parameters
Parameter
Type
Validation
Default value
Description
mediaSource
-
-
Media source information
Return Promise<void>

setCameraCaptureParam()

Set camera capture parameter
import TUIRoomEngine, { TUIMediaMixingManager, TUICameraCaptureMode } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
mediaMixingManager.setCameraCaptureParam('<camera ID>', {
mode: TUICameraCaptureMode.kCameraCaptureManual,
width: 1280,
height: 720,
});
Parameters
Parameter
Type
Validation
Default value
Description
cameraId
string
Non-empty
-
Camera ID
param
-
-
Camera capture parameters
Return

startPublish()

Start local mixing and streaming Note: Streaming can only be successful after entering the room.
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
mediaMixingManager.startPublish();
Return Promise<void>

stopPublish()

Stop local streaming, but still mixing that can be previewed.
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
mediaMixingManager.stopPublish();
Return Promise<void>

updatePublishParams()

Modify local mixing parameters
import TUIRoomEngine, {
TUIMediaMixingManager,
TUIResolutionMode,
TUIVideoResolution
} from '@tencentcloud/tuiroom-engine-electron';

const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
mediaMixingManager.updatePublishParams({
inputSourceList: [], // TUIMediaSource[], can be empty
videoEncoderParams: {
videoResolution: TUIVideoResolution.kVideoResolution_1920_1080,
resMode: TUIResolutionMode.kResolutionMode_Landscape,
videoFps: 30,
videoBitrate: 3000,
minVideoBitrate: 2000,
enableAdjustRes: true
},
canvasColor: 0xc6c6c6, // integer number of RGB
});
Parameters
Parameter
Type
Validation
Default value
Description
params
-
-
Local mixing configuration parameters
Return Promise<void>

Type explaination

TUICameraCaptureParam

Camera capture parameters
Enumeration item
Type
Description
mode
Camera capture mode
width
number
Captured image width
height
number
Captured image height

TUILocalMediaTranscodingParams

Local medix mixing transcoding configuration parameters
Field
Type
Description
inputSourceList
Media source list
videoEncoderParams
Mixed-stream video encoding parameters
canvasColor
number
Mixed-stream video background color, RGB format, default: 0x0

TUIMediaSource

Media source data
Field
Type
Description
sourceType
Media source type
sourceId
string
Unique ID of media source
rect
TUIRect
Coordinate position relative to the local mixing layout
zOrder
number
Display hierarchy. Media sources with higher display hierarchy will cover those with lower display hierarchy, and multiple media sources cannot have the same hierarchy.
rotation
Optional, rotation angle
fillMode
Optional, fill mode
mirrorType
Optional, mirror mode
isSelected
boolean
Optional, default: false, whether the media source is selected

TUIRect

Rectangular coordinate area
Field
Type
Description
left
number
Left line coordinate
top
number
Top line coordinate
right
number
Right line coordinate
bottom
number
Bottom line coordinate

TUIScreenCaptureSourceInfo

Screen/window data
Field
Type
Description
type
Media source type: screen/window
sourceId
number
Unique ID
sourceName
string
Name
isMinimizeWindow
boolean
Whether the window is minimized
isMainScreen
boolean
Whether it is the main screen
rect
TUIRect
Relative coordinate position on the display screen

TUIVideoEncParam

Mixed-stream video encoding parameters
Field
Type
Description
videoResolution
Video resolution
resMode
Video portrait/landscape mode
videoFps
number
Frame rate
videoBitrate
number
Bitrate
minVideoBitrate
number
Minimum bitrate
enableAdjustRes
boolean
Whether to enable adaptive bitrate adjustment, default: true

Enumerations explaination

TUICameraCaptureMode

Camera capture mode
Enumeration item
Type
Description
kCameraResolutionStrategyAuto
number
Automatically adjust capture parameters. The SDK selects appropriate camera output parameters based on the actual performance of the capture device and network conditions, maintaining a balance between device performance and video preview quality.
kCameraResolutionStrategyPerformance
number
Priority on device performance. The SDK selects the closest camera output parameters based on the user's settings for encoder resolution and frame rate, thereby ensuring device performance.
kCameraResolutionStrategyHighQuality
number
Priority on video preview quality. The SDK selects higher camera output parameters to improve the quality of the preview video. In this case, more CPU and memory will be consumed for video preprocessing.
kCameraCaptureManual
number
Allows users to set the local camera capture video width and height.

TUIMediaFillMode

Media displaying Fill mode
Enumeration item
Type
Description
kMediaFillMode_Fill
number
The media fills the display area, and the parts of the video that exceed the display window will be cut off. The picture content may not be displayed completely
kMediaFillMode_Fit
number
The long side of the media fills the display area, and the short side area will be filled with black. The picture content is displayed completely

TUIMediaMirrorType

Mirror mode
Enumeration item
Type
Description
kMediaMirrorType_Auto
number
Automatic mode. During local preview, the front camera is mirrored, and the rear camera is not mirrored
kMediaMirrorType_Enable
number
Enable mirror
kMediaMirrorType_Disable
number
Disable mirror

TUIMediaRotation

Rotation angle
Enumeration item
Type
Description
kMediaRotation0
number
Rotate 0 degrees
kMediaRotation90
number
Rotate 90 degrees
kMediaRotation180
number
Rotate 180 degrees
kMediaRotation270
number
Rotate 270 degrees

TUIMediaSourceType

Media source type
Enumeration item
Type
Description
kCamera
number
Camera
kScreen
number
Screen/Window
kImage
number
Image

TUIResolutionMode

Video resolution mode
Enumeration item
Type
Description
kResolutionMode_Landscape
number
Landscape mode
kResolutionMode_Portrait
number
Portrait mode

TUIVideoResolution

Video resolution. Only landscape resolutions are defined here. If you want to use a portrait resolution such as 360 × 640, you need to specify TUIResolutionMode as Portrait at the same time.
Enumeration item
Type
Description
kVideoResolution_120_120
number
Recommended bitrate (VideoCall) 80kbps; Recommended bitrate (LIVE) 120kbps
kVideoResolution_160_160
number
Recommended bitrate (VideoCall) 100kbps; Recommended bitrate (LIVE) 150kbps
kVideoResolution_270_270
number
Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps
kVideoResolution_480_480
number
Recommended bitrate (VideoCall) 350kbps; Recommended bitrate (LIVE) 500kbps
kVideoResolution_160_120
number
Recommended bitrate (VideoCall) 100kbps; Recommended bitrate (LIVE) 150kbps
kVideoResolution_240_180
number
Recommended bitrate (VideoCall) 150kbps; Recommended bitrate (LIVE) 250kbps
kVideoResolution_280_210
number
Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps
kVideoResolution_320_240
number
Recommended bitrate (VideoCall) 250kbps; Recommended bitrate (LIVE) 375kbps
kVideoResolution_400_300
number
Recommended bitrate (VideoCall) 300kbps; Recommended bitrate (LIVE) 450kbps
kVideoResolution_480_360
number
Recommended bitrate (VideoCall) 400kbps; Recommended bitrate (LIVE) 600kbps
kVideoResolution_640_480
number
Recommended bitrate (VideoCall) 600kbps; Recommended bitrate (LIVE) 900kbps
kVideoResolution_960_720
number
Recommended bitrate (VideoCall) 1000kbps; Recommended bitrate (LIVE) 1500kbps
kVideoResolution_160_90
number
Recommended bitrate (VideoCall) 150kbps; Recommended bitrate (LIVE) 250kbps
kVideoResolution_256_144
number
Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps
kVideoResolution_320_180
number
Recommended bitrate (VideoCall) 250kbps; Recommended bitrate (LIVE) 400kbps
kVideoResolution_480_270
number
Recommended bitrate (VideoCall) 350kbps; Recommended bitrate (LIVE) 550kbps
kVideoResolution_640_360
number
Recommended bitrate (VideoCall) 500kbps; Recommended bitrate (LIVE) 900kbps
kVideoResolution_960_540
number
Recommended bitrate (VideoCall) 850kbps; Recommended bitrate (LIVE) 1300kbps
kVideoResolution_1280_720
number
Recommended bitrate (VideoCall) 1200kbps; Recommended bitrate (LIVE) 1800kbps
kVideoResolution_1920_1080
number
Recommended bitrate (VideoCall) 2000kbps; Recommended bitrate (LIVE) 3000kbpy