Support two different screen sharing solutions on the iOS platform:
in-app sharing
Share only the current App's screen. This feature requires an iOS 13 and above versions operating system to support. Since it is unable to share screen content outside the current App, it is suitable for scenarios with high privacy protection requirements.
cross-app sharing
Based on Apple's Replaykit solution, it can share the screen content of the entire system. However, the current App needs to provide an additional Extension component. Therefore, the steps required to implement are relatively more than in-app sharing.
Supported Platforms
iOS
Android
Mac OS
Windows
Electron
Google Chrome Browser
✓
✓
✓
✓
✓
✓
In-App Sharing
The solution for in-app sharing is super simple. Just call the interface startScreenCapture provided by RoomEngine and pass in the input parameter appgroup.
Meanwhile, you can call updateVideoQualityEx to modify the encoding parameters. The encoding parameters we recommend for iOS screen sharing are:
Parameter Item
Parameter Name
Regular Recommended Value
Text Teaching Scenario
Resolution
videoResolution
1280 × 720
1920 × 1080
Frame Rate
fps
10 FPS
8 FPS
Maximum Bit Rate
bitrate
1600 kbps
2000 kbps
Since the content of screen sharing generally does not change severely, setting a relatively high FPS is not economic. 10 FPS is recommended.
If you need higher video quality, increase the FPS.
If the screen content you want to share contains a large amount of text, you can appropriately increase the resolution and bitrate settings.
The highest bitrate (videoBitrate) refers to the maximum output bitrate when the picture changes dramatically. If the screen content changes less, the actual encoding bitrate will be relatively low.
Example:
importRTCRoomEngine
let roomEngine =TUIRoomEngine.sharedInstance()
roomEngine.startScreenCapture(appGroup:"")
let params =TUIRoomVideoEncoderParams()
params.fps =10//replace with the actual value you need
Cross-application screen sharing on the iOS system requires adding the Broadcast Upload Extension screen recording process to cooperate with the main App process in starting push. The Extension screen recording process is created by the system when screen recording is needed and is responsible for receiving screen images collected by the system. Therefore, it is required:
1. Create App Group and configure it in XCode (optional). The purpose of this step is to enable cross-process communication between the Extension screen recording process and the main app process.
2. In your project, create a new Target for Broadcast Upload Extension and integrate TXLiteAVSDK_ReplayKitExt.framework, which is specially customized for the expansion module in the SDK compression package, in it.
3. Integrate with the receiving logic of the main App, allowing the main App to wait for screen recording data from Broadcast Upload Extension.
Notes:
If you skip step 1, that is, do not configure the App Group (pass nil through the API), the screen-sharing feature can still run, but its stability will be compromised. Therefore, although there are many steps involved, please configure the correct App Group as much as possible to ensure the stability of the screen-sharing feature.
Step 1: Create an App Group
Log in with your account https://developer.apple.com/ and perform the following operations. Note that you need to re-download the corresponding Provisioning Profile after completion.
1. 1. Click Certificates, IDs & Profiles.
2. Click the plus sign on the right interface.
3. Select App Groups, click Continue.
4. Fill in Description and Identifier in the pop-up form. Among them, the Identifier needs to be passed into the corresponding AppGroup parameter in the API. Once completed, click Continue.
5. Return to the Identifier page, select App IDs from the menu on the upper left, and then click your App ID (the AppIDs of the main App and the Extension need the same configuration).
6. Select App Groups and click Edit.
7. Select the App Group you created earlier in the pop-up form, click Continue to return to the edit page, and click Save to save.
8. Re-download the Provisioning Profile and configure it to XCode.
Step 2: Create a Broadcast Upload Extension
1. Click sequentially File, New, Target... in the Xcode menu, and select Broadcast Upload Extension.
2. Fill in relevant information in the pop-up dialog box. Do not check Include UI Extension. Click Finish to complete the creation.
3. Drag TXLiteAVSDK_ReplayKitExt.framework in the downloaded SDK compression package to the project, and select the newly created Target.
4. Select the newly added Target, click sequentially + Capability, double-click App Groups, as shown in the figure below:
After the operation is completed, a file named Target Name.entitlements will be generated in the file list. As shown in the figure below, select the file and click the + sign to fill in the App Group from the above steps.
5. Select the Target of the main App, and perform the same handling for the Target of the main App as the above steps.
6. In the newly created Target, Xcode will automatically create a file named "SampleHandler.h". Replace it with the following code. Change APPGROUP in the code to the App Group Identifier created above.
Step 3: Integrating the Receiving Logic of the Main App
Follow the steps below to integrate the receiving logic of the main App. That is, before the user triggers screen sharing, put the main App in the "waiting" status so that it can receive screen recording data from the Broadcast Upload Extension process at any time.
1. Call the startScreenCapture method and input the AppGroup set in Step 1 to put the SDK in the "waiting" status.
2. Wait for the user to trigger screen sharing. If the "trigger button" in Step 4 is not implemented, for screen sharing on an iOS device, the user needs to long-press the screen recording button in the control center. The operation steps are as shown in the figure:
3. By calling the stopScreenCapture api, you can terminate screen sharing at any time.
Step 4: Add a Trigger Button for Screen Sharing (Optional)
As of Step 3, our screen sharing still requires users to manually start by long-pressing the screen recording button in the control center. You can achieve an effect similar to Tencent Meeting, where a single click on a button triggers it, by following the method below:
1. Implemented awakening screen sharing in the TRTCBroadcastExtensionLauncher file. Add it to your project.
2. Place a button on your interface and call the launch function in TRTCBroadcastExtensionLauncher in the button's response function. You can then activate the screen-sharing feature.
// The pop-up on iOS 12 is slow and will crash if you click quickly.
let now =CFAbsoluteTimeGetCurrent()
if now - prevLaunchEventTime <1.0{
return
}
prevLaunchEventTime = now
for view in systemBroacastExtensionPicker.subviews {
iflet button = view as?UIButton{
button.sendActions(for:.allTouchEvents)
break
}
}
}
}
Notes:
Apple added RPSystemBroadcastPickerView in iOS 12.0, which can pop up a launcher from the application for user confirmation to start screen sharing. So far, RPSystemBroadcastPickerView does not support customization of the interface, nor does it have an official triggering method.
The principle behind TRTCBroadcastExtensionLauncher is to traverse the child views of RPSystemBroadcastPickerView to find a UIButton and trigger its click event.
However, this solution is not officially recommended by Apple and may become invalid in the next system update. ThereforeStep 4is just an optional solution. You need to assume the risk of selecting this solution.
Step 5: Screen Sharing in Stream Mixing Mode
1.After you complete the above logic, you need to call the setLiveStreamLayoutInfo API to modify the screen layout.
importRTCRoomEngine
let roomEngine =TUIRoomEngine.sharedInstance()
let layoutManager = roomEngine.getExtension(extensionType:.liveLayoutManager)as?TUILiveLayoutManager
layoutManager.setLiveStreamLayoutInfo(
roomID: roomInfo.roomId,
layoutInfo:""//see below
onSuccess:{
//Modification succeeded
},
onError:{code ,message in
}
)
//Format requirements for layoutInfo:
{
"RoomId":"live_adams",//Replace with your Room ID
// When layouttype is 1000, modify LayoutInfo while modifying VideoEncode. In 9-grid view mode, LayoutInfo will be automatically modified and updated based on resolution by default.
"VideoEncode":{
"Width":1080,
"Height":1920,
},
"LayoutMode":0,// 0-9 built-in layout templates, 1000 custom layout, LayoutInfo can only be modified when it is 1000, 0 is nine-square grid
"LayoutInfo":{
"LayoutList":[
{
"LocationX":0,// Specific values
"LocationY":0,// Specific values
"ImageWidth":1080,// Specific values
"ImageHeight":960,// Specific values
"ZOrder":0,// Level
"StreamType":0,// 0 for camera, 1 for screen sharing
"Member_Account":"admin001",// Replace with your userId
"BackgroundImageUrl":"ImageUrl",
"RoomId":"live_adams",//Replace with your RoomId
"BackgroundColor":"0x1F212C"
},
{
"LocationX":0,// Specific values
"LocationY":960,// Specific values
"ImageWidth":1080,// Specific values
"ImageHeight":960,// Specific values
"ZOrder":0,// Level
"StreamType":0,// 0 for camera, 1 for screen sharing
"Member_Account":"admin001",// Replace with your UserId
"BackgroundImageUrl":"ImageUrl",
"RoomId":"live_adams",//Replace with your RoomId
"BackgroundColor":"0x1F212C"
}
],
"MaxUserLayout":{
"LocationX":0,// Specific values
"LocationY":0,// Specific values
"ImageWidth":1080,// Specific values
"ImageHeight":1920,// Specific values
"ZOrder":0,// Level
"StreamType":0,// 0 for camera, 1 for screen sharing
"Member_Account":"admin001",// Replace with your userId
"BackgroundImageUrl":"ImageUrl",
"RoomId":"live_adams",// Replace with your roomId
"BackgroundColor":"0x1F212C"
}
}
Step 6: Stop Sharing in Stream Mixing Mode
1. When you want to stop screen sharing and modify the screen layout back, you need to call the stopScreenCapture API to close and call the setLiveStreamLayoutInfo API to recover the screen layout.
importRTCRoomEngine
let roomEngine =TUIRoomEngine.sharedInstance()
let layoutManager = roomEngine.getExtension(extensionType:.liveLayoutManager)as?TUILiveLayoutManager
layoutManager.setLiveStreamLayoutInfo(
roomID: roomInfo.roomId,
layoutInfo:"",//see below
onSuccess:{},
onError:{ code ,message in
}
)
//When you want to restore the default grid layout, the layoutInfo parameter you need to input is as follows:
{
"RoomId":"live_12121",// Replace with the actual room number
"VideoEncode":{
"Width":1080,
"Height":1920
},
"LayoutMode":0
}
Viewing Screen Sharing
When a user plays screen sharing, you can listen to onUserVideoStateChanged in TUIRoomObserver to perceive the screen share stream.
Users who want to view screen sharing can start rendering the mainstream picture of remote users through the startRemoteView API.
Example:
importRTCRoomEngine
extensionScreenSharedController:TUIRoomObserver{// Replace with your specific business class, here is just an example