• 서비스
  • 가격
  • 리소스
  • 기술지원
이 페이지는 현재 영어로만 제공되며 한국어 버전은 곧 제공될 예정입니다. 기다려 주셔서 감사드립니다.

Screen Sharing

Support two different screen sharing solutions on the iOS platform:
in-app sharing
Share only the current App's screen. This feature requires an iOS 13 and above versions operating system to support. Since it is unable to share screen content outside the current App, it is suitable for scenarios with high privacy protection requirements.
cross-app sharing
Based on Apple's Replaykit solution, it can share the screen content of the entire system. However, the current App needs to provide an additional Extension component. Therefore, the steps required to implement are relatively more than in-app sharing.

Supported Platforms

iOS
Android
Mac OS
Windows
Electron
Google Chrome Browser

In-App Sharing

The solution for in-app sharing is super simple. Just call the interface startScreenCapture provided by RoomEngine and pass in the input parameter appgroup.
Meanwhile, you can call updateVideoQualityEx to modify the encoding parameters. The encoding parameters we recommend for iOS screen sharing are:
Parameter Item
Parameter Name
Regular Recommended Value
Text Teaching Scenario
Resolution
videoResolution
1280 × 720
1920 × 1080
Frame Rate
fps
10 FPS
8 FPS
Maximum Bit Rate
bitrate
1600 kbps
2000 kbps
Since the content of screen sharing generally does not change severely, setting a relatively high FPS is not economic. 10 FPS is recommended.
If you need higher video quality, increase the FPS.
If the screen content you want to share contains a large amount of text, you can appropriately increase the resolution and bitrate settings.
The highest bitrate (videoBitrate) refers to the maximum output bitrate when the picture changes dramatically. If the screen content changes less, the actual encoding bitrate will be relatively low.
Example:
import RTCRoomEngine

let roomEngine = TUIRoomEngine.sharedInstance()

roomEngine.startScreenCapture(appGroup: "")

let params = TUIRoomVideoEncoderParams()
params.fps = 10 //replace with the actual value you need
params.resolutionMode =.portrait //portrait resolution
params.bitrate =.1600 //replace here with the actual value you need
params.videoResolution =.quality720P //replace here with the actual value you need

roomEngine.updateVideoQualityEx(streamType: .screenStream, params: params)

Cross-App Sharing

Cross-application screen sharing on the iOS system requires adding the Broadcast Upload Extension screen recording process to cooperate with the main App process in starting push. The Extension screen recording process is created by the system when screen recording is needed and is responsible for receiving screen images collected by the system. Therefore, it is required:
1. Create App Group and configure it in XCode (optional). The purpose of this step is to enable cross-process communication between the Extension screen recording process and the main app process.
2. In your project, create a new Target for Broadcast Upload Extension and integrate TXLiteAVSDK_ReplayKitExt.framework, which is specially customized for the expansion module in the SDK compression package, in it.
3. Integrate with the receiving logic of the main App, allowing the main App to wait for screen recording data from Broadcast Upload Extension.
Notes:
If you skip step 1, that is, do not configure the App Group (pass nil through the API), the screen-sharing feature can still run, but its stability will be compromised. Therefore, although there are many steps involved, please configure the correct App Group as much as possible to ensure the stability of the screen-sharing feature.

Step 1: Create an App Group

Log in with your account https://developer.apple.com/ and perform the following operations. Note that you need to re-download the corresponding Provisioning Profile after completion.
1. 1. Click Certificates, IDs & Profiles.
2. Click the plus sign on the right interface.
3. Select App Groups, click Continue.
4. Fill in Description and Identifier in the pop-up form. Among them, the Identifier needs to be passed into the corresponding AppGroup parameter in the API. Once completed, click Continue.



5. Return to the Identifier page, select App IDs from the menu on the upper left, and then click your App ID (the AppIDs of the main App and the Extension need the same configuration).
6. Select App Groups and click Edit.
7. Select the App Group you created earlier in the pop-up form, click Continue to return to the edit page, and click Save to save.



8. Re-download the Provisioning Profile and configure it to XCode.

Step 2: Create a Broadcast Upload Extension

1. Click sequentially File, New, Target... in the Xcode menu, and select Broadcast Upload Extension.
2. Fill in relevant information in the pop-up dialog box. Do not check Include UI Extension. Click Finish to complete the creation.
3. Drag TXLiteAVSDK_ReplayKitExt.framework in the downloaded SDK compression package to the project, and select the newly created Target.
4. Select the newly added Target, click sequentially + Capability, double-click App Groups, as shown in the figure below:

AddCapability


After the operation is completed, a file named Target Name.entitlements will be generated in the file list. As shown in the figure below, select the file and click the + sign to fill in the App Group from the above steps.

AddGroup


5. Select the Target of the main App, and perform the same handling for the Target of the main App as the above steps.
6. In the newly created Target, Xcode will automatically create a file named "SampleHandler.h". Replace it with the following code. Change APPGROUP in the code to the App Group Identifier created above.
import ReplayKit
import TXLiteAVSDK_ReplayKitExt

let APPGROUP = ""

class SampleHandler: RPBroadcastSampleHandler, TXReplayKitExtDelegate {

let recordScreenKey = Notification.Name.init("TRTCRecordScreenKey")

override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) {
if let setupInfo = setupInfo {
}
TXReplayKitExt.sharedInstance().setup(withAppGroup: APPGROUP, delegate: self)
}
override func broadcastPaused() {
// Add logic for resource release or status save during suspension
}

override func broadcastResumed() {
// Can add logic for reinitializing resources when recovering
}

override func broadcastFinished(){
// User has requested to finish the broadcast.
TXReplayKitExt.sharedInstance().broadcastFinished()
}

func broadcastFinished(_ broadcast: TXReplayKitExt, reason: TXReplayKitExtReason) {
var tip = ""
switch reason {
case TXReplayKitExtReason.requestedByMain:
tip = "Screen sharing has ended"
break
case TXReplayKitExtReason.disconnected:
tip = "Application disconnected"
break
case TXReplayKitExtReason.versionMismatch:
tip = "Integration error (SDK version mismatch)"
break
default:
break
}

let error = NSError(domain: NSStringFromClass(self.classForCoder), code: 0, userInfo: [NSLocalizedFailureReasonErrorKey:tip])
finishBroadcastWithError(error)
}

override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
switch sampleBufferType {
case RPSampleBufferType.video:
TXReplayKitExt
.sharedInstance()
.send(sampleBuffer, with: .video)
break
case RPSampleBufferType.audioApp:
// You can process application audio here
break
case RPSampleBufferType.audioMic:
// You can process microphone audio here
break
@unknown default:
let error = "Unknown sampling buffer type: \(sampleBufferType)"
finishBroadcastWithError(NSError(domain: error, code: -1, userInfo: nil))
}
}
}

Step 3: Integrating the Receiving Logic of the Main App

Follow the steps below to integrate the receiving logic of the main App. That is, before the user triggers screen sharing, put the main App in the "waiting" status so that it can receive screen recording data from the Broadcast Upload Extension process at any time.
1. Call the startScreenCapture method and input the AppGroup set in Step 1 to put the SDK in the "waiting" status.
2. Wait for the user to trigger screen sharing. If the "trigger button" in Step 4 is not implemented, for screen sharing on an iOS device, the user needs to long-press the screen recording button in the control center. The operation steps are as shown in the figure:
3. By calling the stopScreenCapture api, you can terminate screen sharing at any time.
Example:
import RTCRoomEngine

TUIRoomEngine.sharedInstance().startScreenCapture(appGroup: "")
TUIRoomEngine.sharedInstance().stopScreenCapture()

Step 4: Add a Trigger Button for Screen Sharing (Optional)

As of Step 3, our screen sharing still requires users to manually start by long-pressing the screen recording button in the control center. You can achieve an effect similar to Tencent Meeting, where a single click on a button triggers it, by following the method below:
1. Implemented awakening screen sharing in the TRTCBroadcastExtensionLauncher file. Add it to your project.
2. Place a button on your interface and call the launch function in TRTCBroadcastExtensionLauncher in the button's response function. You can then activate the screen-sharing feature.
Example:
@objc private func buttonTapped() {
TRTCBroadcastExtensionLauncher.sharedInstance.launch()
}
import UIKit
import TXLiteAVSDK_ReplayKitExt

class TRTCBroadcastExtensionLauncher: NSObject {

var systemBroacastExtensionPicker = RPSystemBroadcastPickerView()
var prevLaunchEventTime : CFTimeInterval = 0

static let sharedInstance = TRTCBroadcastExtensionLauncher()

override init() {
super.init()
let picker = RPSystemBroadcastPickerView(frame: CGRect(x: 0, y: 0, width: 44, height: 44))
picker.showsMicrophoneButton = false
picker.autoresizingMask = [.flexibleTopMargin, .flexibleRightMargin]
systemBroacastExtensionPicker = picker

if let pluginPath = Bundle.main.builtInPlugInsPath,
let contents = try? FileManager.default.contentsOfDirectory(atPath: pluginPath) {

for content in contents where content.hasSuffix(".appex") {
guard let bundle = Bundle(path: URL(fileURLWithPath: pluginPath).appendingPathComponent(content).path),
let identifier : String = (bundle.infoDictionary?["NSExtension"] as? [String:Any])? ["NSExtensionPointIdentifier"] as? String
else {
continue
}
if identifier == "com.apple.broadcast-services-upload" {
picker.preferredExtension = bundle.bundleIdentifier
break
}
}
}
}

static func launch() {
TRTCBroadcastExtensionLauncher.sharedInstance.launch()
}

func launch() {
// The pop-up on iOS 12 is slow and will crash if you click quickly.
let now = CFAbsoluteTimeGetCurrent()
if now - prevLaunchEventTime < 1.0 {
return
}
prevLaunchEventTime = now

for view in systemBroacastExtensionPicker.subviews {
if let button = view as? UIButton {
button.sendActions(for: .allTouchEvents)
break
}
}
}
}

Notes:
Apple added RPSystemBroadcastPickerView in iOS 12.0, which can pop up a launcher from the application for user confirmation to start screen sharing. So far, RPSystemBroadcastPickerView does not support customization of the interface, nor does it have an official triggering method.
The principle behind TRTCBroadcastExtensionLauncher is to traverse the child views of RPSystemBroadcastPickerView to find a UIButton and trigger its click event.
However, this solution is not officially recommended by Apple and may become invalid in the next system update. Therefore Step 4 is just an optional solution. You need to assume the risk of selecting this solution.

Step 5: Screen Sharing in Stream Mixing Mode

1.After you complete the above logic, you need to call the setLiveStreamLayoutInfo API to modify the screen layout.
import RTCRoomEngine

let roomEngine = TUIRoomEngine.sharedInstance()
let layoutManager = roomEngine.getExtension(extensionType: .liveLayoutManager) as? TUILiveLayoutManager

layoutManager.setLiveStreamLayoutInfo(
roomID: roomInfo.roomId,
layoutInfo: "" //see below
onSuccess: {
//Modification succeeded
},
onError: {code ,message in
}
)
//Format requirements for layoutInfo:
{
"RoomId": "live_adams", //Replace with your Room ID
// When layouttype is 1000, modify LayoutInfo while modifying VideoEncode. In 9-grid view mode, LayoutInfo will be automatically modified and updated based on resolution by default.
"VideoEncode": {
"Width": 1080,
"Height": 1920,
},
"LayoutMode": 0, // 0-9 built-in layout templates, 1000 custom layout, LayoutInfo can only be modified when it is 1000, 0 is nine-square grid
"LayoutInfo":{
"LayoutList": [
{
"LocationX": 0, // Specific values
"LocationY": 0, // Specific values
"ImageWidth": 1080, // Specific values
"ImageHeight": 960, // Specific values
"ZOrder": 0, // Level
"StreamType": 0, // 0 for camera, 1 for screen sharing
"Member_Account": "admin001", // Replace with your userId
"BackgroundImageUrl": "ImageUrl",
"RoomId":"live_adams", //Replace with your RoomId
"BackgroundColor":"0x1F212C"
},
{
"LocationX": 0, // Specific values
"LocationY": 960, // Specific values
"ImageWidth": 1080, // Specific values
"ImageHeight": 960, // Specific values
"ZOrder": 0, // Level
"StreamType": 0, // 0 for camera, 1 for screen sharing
"Member_Account": "admin001", // Replace with your UserId
"BackgroundImageUrl": "ImageUrl",
"RoomId":"live_adams", //Replace with your RoomId
"BackgroundColor":"0x1F212C"
}
],
"MaxUserLayout": {
"LocationX": 0, // Specific values
"LocationY": 0, // Specific values
"ImageWidth": 1080, // Specific values
"ImageHeight": 1920, // Specific values
"ZOrder": 0, // Level
"StreamType": 0, // 0 for camera, 1 for screen sharing
"Member_Account": "admin001", // Replace with your userId
"BackgroundImageUrl": "ImageUrl",
"RoomId":"live_adams", // Replace with your roomId
"BackgroundColor":"0x1F212C"
}
}

Step 6: Stop Sharing in Stream Mixing Mode

1. When you want to stop screen sharing and modify the screen layout back, you need to call the stopScreenCapture API to close and call the setLiveStreamLayoutInfo API to recover the screen layout.
import RTCRoomEngine

let roomEngine = TUIRoomEngine.sharedInstance()
let layoutManager = roomEngine.getExtension(extensionType: .liveLayoutManager) as? TUILiveLayoutManager

layoutManager.setLiveStreamLayoutInfo(
roomID: roomInfo.roomId,
layoutInfo: "",//see below
onSuccess: {},
onError: { code ,message in
}
)
//When you want to restore the default grid layout, the layoutInfo parameter you need to input is as follows:
{
"RoomId": "live_12121", // Replace with the actual room number
"VideoEncode": {
"Width": 1080,
"Height": 1920
},
"LayoutMode": 0
}

Viewing Screen Sharing

When a user plays screen sharing, you can listen to onUserVideoStateChanged in TUIRoomObserver to perceive the screen share stream.
Users who want to view screen sharing can start rendering the mainstream picture of remote users through the startRemoteView API.
Example:
import RTCRoomEngine

extension ScreenSharedController: TUIRoomObserver { // Replace with your specific business class, here is just an example
func onUserVideoStateChanged(userId: String, streamType: TUIVideoStreamType, hasVideo: Bool, reason: TUIChangeReason) {
}
}