TUIMediaMixingManager
Introduction
TUIMediaMixingManager
is a local multimedia mixing manager that supports mixing multiple local video streams into one video stream for streaming.Installation:
// npmnpm i @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3 --save// pnpmpnpm i @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3 --save// yarnyarn add @tencentcloud/tuiroom-engine-electron@2.4.0-alpha.3
Example code:
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();
API
Method | Description |
Set the program path for the local mixing service process | |
Set the local mixing video preview window and area | |
Add a multimedia source | |
Remove a multimedia source | |
Update a multimedia source | |
Set camera capture parameters | |
Start local mixing and streaming | |
Stop local mixing and streaming | |
Modify local mixing parameters |
Type Definitions
Type | Description |
Camera capture parameters | |
Local mixing configuration parameters | |
Media source data | |
Rectangular coordinate area, including left, top, right, bottom properties | |
Screen/window data | |
Mixed video encoding parameters |
Enumerations
Enum | Description |
Camera capture mode | |
Media source fill mode | |
Media source mirror type | |
Media source rotation angle | |
Media source type | |
Resolution mode (landscape resolution/vertical resolution) | |
Video resolution |
API explaination
setMediaServerPath()
Set the program path for the local mixing service process. After creating an instance object, this method must be called first for other methods to take effect.
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();async function initMediaServer() {const isPackaged = location.href.startsWith("file"); // Same with Electron `app.isPackaged`const resourcesPath = (globalThis as any).process.resourcesPath;let mediaServerPath = '';if (isPackaged) {mediaServerPath = `${resourcesPath}\\liteav_media_server.exe`;} else {const appPath = ''; // Value from Electron main process: app.getAppPath()mediaServerPath = `${appPath}\\node_modules\\trtc-electron-sdk\\build\\Release\\liteav_media_server.exe`;}mediaMixingManager.setMediaServerPath(mediaServerPath);}initMediaServer();
Parameters
Parameter | Type | Validation | Default value | Description |
path | string | - | - | Program path for the local mixing service process |
Return
Promise
<void
>setDisplayParams()
Set the local mixing video preview window and area
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();// Value from Electron main BrowserWindow.getNativeWindowHandle()const nativeWindowHandle = new Uint8Array(8);const previewDOMNode = document.getElementByID("preview-media-mixing");const clientRect = previewDOMNode.getBoundingClientRect();mediaMixingManager.setDisplayParams(window.nativeWindowHandle, {left: clientRect.left * window.devicePixelRatio, // Notice: window.devicePixelRatioright: clientRect.right * window.devicePixelRatio, // Notice: window.devicePixelRatiotop: clientRect.top * window.devicePixelRatio, // Notice: window.devicePixelRatiobottom: clientRect.bottom * window.devicePixelRatio, // Notice: window.devicePixelRatio});
Parameters
Parameter | Type | Validation | Default value | Description |
windowID | Uint8Array | - | - | Mixed video preview window ID, obtained through Electron API BrowserView.getNativeWindowHandle() |
rect | - | - | Mixed video preview area position and size |
Return
void
addMediaSource()
Add media source
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();const mediaSource:TUIMediaSource = {sourceType: TUIMediaSourceType.kCamera,sourceId: '', // camera ID, screen ID, window ID or image pathzOrder: 1, // should be unique among all media sourcesrect: {left: 0,top: 0,right: 640,bottom: 360},};mediaMixingManager.addMediaSource(mediaSource);
Parameters
Parameter | Type | Validation | Default value | Description |
mediaSource | - | - | Media source information |
Return
Promise
<TUIRect> Default position in mixing video imageremoveMediaSource()
Remove media source
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();const mediaSource:TUIMediaSource = {sourceType: TUIMediaSourceType.kCamera, // requiredsourceId: '', // requiredzOrder: 1, // any valid valuerect: { // any valid valueleft: 0,top: 0,right: 640,bottom: 360},};mediaMixingManager.removeMediaSource(mediaSource);
Parameters
Parameter | Type | Validation | Default value | Description |
mediaSource | - | - | Media source information |
Return
Promise
<void
>updateMediaSource()
Update media source
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();const mediaSource:TUIMediaSource = {sourceType: TUIMediaSourceType.kCamera, // requiredsourceId: '', // requiredzOrder: 1, // should be unique among all media sourcesrect: { // any valid valueleft: 0,top: 0,right: 640,bottom: 360},};mediaMixingManager.updateMediaSource(mediaSource);
Parameters
Parameter | Type | Validation | Default value | Description |
mediaSource | - | - | Media source information |
Return
Promise
<void
>setCameraCaptureParam()
Set camera capture parameter
import TUIRoomEngine, { TUIMediaMixingManager, TUICameraCaptureMode } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();mediaMixingManager.setCameraCaptureParam('<camera ID>', {mode: TUICameraCaptureMode.kCameraCaptureManual,width: 1280,height: 720,});
Parameters
Parameter | Type | Validation | Default value | Description |
cameraId | string | Non-empty | - | Camera ID |
param | - | - | Camera capture parameters |
Return
startPublish()
Start local mixing and streaming
Note: Streaming can only be successful after entering the room.
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();mediaMixingManager.startPublish();
Return
Promise
<void
>stopPublish()
Stop local streaming, but still mixing that can be previewed.
import TUIRoomEngine, { TUIMediaMixingManager } from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();mediaMixingManager.stopPublish();
Return
Promise
<void
>updatePublishParams()
Modify local mixing parameters
import TUIRoomEngine, {TUIMediaMixingManager,TUIResolutionMode,TUIVideoResolution} from '@tencentcloud/tuiroom-engine-electron';const mediaMixingManager: TUIMediaMixingManager = TUIRoomEngine.getMediaMixingManager();mediaMixingManager.updatePublishParams({inputSourceList: [], // TUIMediaSource[], can be emptyvideoEncoderParams: {videoResolution: TUIVideoResolution.kVideoResolution_1920_1080,resMode: TUIResolutionMode.kResolutionMode_Landscape,videoFps: 30,videoBitrate: 3000,minVideoBitrate: 2000,enableAdjustRes: true},canvasColor: 0xc6c6c6, // integer number of RGB});
Parameters
Parameter | Type | Validation | Default value | Description |
params | - | - | Local mixing configuration parameters |
Return
Promise
<void
>Type explaination
TUICameraCaptureParam
Camera capture parameters
Enumeration item | Type | Description |
mode | Camera capture mode | |
width | number | Captured image width |
height | number | Captured image height |
TUILocalMediaTranscodingParams
Local medix mixing transcoding configuration parameters
Field | Type | Description |
inputSourceList | Media source list | |
videoEncoderParams | Mixed-stream video encoding parameters | |
canvasColor | number | Mixed-stream video background color, RGB format, default: 0x0 |
TUIMediaSource
Media source data
Field | Type | Description |
sourceType | Media source type | |
sourceId | string | Unique ID of media source |
rect | Coordinate position relative to the local mixing layout | |
zOrder | number | Display hierarchy. Media sources with higher display hierarchy will cover those with lower display hierarchy, and multiple media sources cannot have the same hierarchy. |
rotation | Optional, rotation angle | |
fillMode | Optional, fill mode | |
mirrorType | Optional, mirror mode | |
isSelected | boolean | Optional, default: false, whether the media source is selected |
TUIRect
Rectangular coordinate area
Field | Type | Description |
left | number | Left line coordinate |
top | number | Top line coordinate |
right | number | Right line coordinate |
bottom | number | Bottom line coordinate |
TUIScreenCaptureSourceInfo
Screen/window data
Field | Type | Description |
type | Media source type: screen/window | |
sourceId | number | Unique ID |
sourceName | string | Name |
isMinimizeWindow | boolean | Whether the window is minimized |
isMainScreen | boolean | Whether it is the main screen |
rect | Relative coordinate position on the display screen |
TUIVideoEncParam
Mixed-stream video encoding parameters
Field | Type | Description |
videoResolution | Video resolution | |
resMode | Video portrait/landscape mode | |
videoFps | number | Frame rate |
videoBitrate | number | Bitrate |
minVideoBitrate | number | Minimum bitrate |
enableAdjustRes | boolean | Whether to enable adaptive bitrate adjustment, default: true |
Enumerations explaination
TUICameraCaptureMode
Camera capture mode
Enumeration item | Type | Description |
kCameraResolutionStrategyAuto | number | Automatically adjust capture parameters. The SDK selects appropriate camera output parameters based on the actual performance of the capture device and network conditions, maintaining a balance between device performance and video preview quality. |
kCameraResolutionStrategyPerformance | number | Priority on device performance. The SDK selects the closest camera output parameters based on the user's settings for encoder resolution and frame rate, thereby ensuring device performance. |
kCameraResolutionStrategyHighQuality | number | Priority on video preview quality. The SDK selects higher camera output parameters to improve the quality of the preview video. In this case, more CPU and memory will be consumed for video preprocessing. |
kCameraCaptureManual | number | Allows users to set the local camera capture video width and height. |
TUIMediaFillMode
Media displaying Fill mode
Enumeration item | Type | Description |
kMediaFillMode_Fill | number | The media fills the display area, and the parts of the video that exceed the display window will be cut off. The picture content may not be displayed completely |
kMediaFillMode_Fit | number | The long side of the media fills the display area, and the short side area will be filled with black. The picture content is displayed completely |
TUIMediaMirrorType
Mirror mode
Enumeration item | Type | Description |
kMediaMirrorType_Auto | number | Automatic mode. During local preview, the front camera is mirrored, and the rear camera is not mirrored |
kMediaMirrorType_Enable | number | Enable mirror |
kMediaMirrorType_Disable | number | Disable mirror |
TUIMediaRotation
Rotation angle
Enumeration item | Type | Description |
kMediaRotation0 | number | Rotate 0 degrees |
kMediaRotation90 | number | Rotate 90 degrees |
kMediaRotation180 | number | Rotate 180 degrees |
kMediaRotation270 | number | Rotate 270 degrees |
TUIMediaSourceType
Media source type
Enumeration item | Type | Description |
kCamera | number | Camera |
kScreen | number | Screen/Window |
kImage | number | Image |
TUIResolutionMode
Video resolution mode
Enumeration item | Type | Description |
kResolutionMode_Landscape | number | Landscape mode |
kResolutionMode_Portrait | number | Portrait mode |
TUIVideoResolution
Video resolution. Only landscape resolutions are defined here. If you want to use a portrait resolution such as 360 × 640, you need to specify TUIResolutionMode as Portrait at the same time.
Enumeration item | Type | Description |
kVideoResolution_120_120 | number | Recommended bitrate (VideoCall) 80kbps; Recommended bitrate (LIVE) 120kbps |
kVideoResolution_160_160 | number | Recommended bitrate (VideoCall) 100kbps; Recommended bitrate (LIVE) 150kbps |
kVideoResolution_270_270 | number | Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps |
kVideoResolution_480_480 | number | Recommended bitrate (VideoCall) 350kbps; Recommended bitrate (LIVE) 500kbps |
kVideoResolution_160_120 | number | Recommended bitrate (VideoCall) 100kbps; Recommended bitrate (LIVE) 150kbps |
kVideoResolution_240_180 | number | Recommended bitrate (VideoCall) 150kbps; Recommended bitrate (LIVE) 250kbps |
kVideoResolution_280_210 | number | Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps |
kVideoResolution_320_240 | number | Recommended bitrate (VideoCall) 250kbps; Recommended bitrate (LIVE) 375kbps |
kVideoResolution_400_300 | number | Recommended bitrate (VideoCall) 300kbps; Recommended bitrate (LIVE) 450kbps |
kVideoResolution_480_360 | number | Recommended bitrate (VideoCall) 400kbps; Recommended bitrate (LIVE) 600kbps |
kVideoResolution_640_480 | number | Recommended bitrate (VideoCall) 600kbps; Recommended bitrate (LIVE) 900kbps |
kVideoResolution_960_720 | number | Recommended bitrate (VideoCall) 1000kbps; Recommended bitrate (LIVE) 1500kbps |
kVideoResolution_160_90 | number | Recommended bitrate (VideoCall) 150kbps; Recommended bitrate (LIVE) 250kbps |
kVideoResolution_256_144 | number | Recommended bitrate (VideoCall) 200kbps; Recommended bitrate (LIVE) 300kbps |
kVideoResolution_320_180 | number | Recommended bitrate (VideoCall) 250kbps; Recommended bitrate (LIVE) 400kbps |
kVideoResolution_480_270 | number | Recommended bitrate (VideoCall) 350kbps; Recommended bitrate (LIVE) 550kbps |
kVideoResolution_640_360 | number | Recommended bitrate (VideoCall) 500kbps; Recommended bitrate (LIVE) 900kbps |
kVideoResolution_960_540 | number | Recommended bitrate (VideoCall) 850kbps; Recommended bitrate (LIVE) 1300kbps |
kVideoResolution_1280_720 | number | Recommended bitrate (VideoCall) 1200kbps; Recommended bitrate (LIVE) 1800kbps |
kVideoResolution_1920_1080 | number | Recommended bitrate (VideoCall) 2000kbps; Recommended bitrate (LIVE) 3000kbpy |