In the lyrics synchronization scheme, the actions of three different roles are as follows:
Lead Singer
Chorus
Audience
NTP time calibration
Turn on black frame compensation
Send SEI message
Local lyrics synchronization
Update lyrics widget
NTP time calibration
Local lyrics synchronization
Update lyrics widget
NTP time calibration
Receive SEI message
Update lyrics widget
The lead singer and chorus update the lyrics progress locally according to the synchronized song playback progress; the audience needs to receive the SEI message sent by the lead singer, which contains the latest lyrics progress, to update the local lyrics progress.
Timing diagram
The lyrics synchronization timing can be mainly divided into three parts: NTP time calibration, turning on black frame compensation, and local and remote lyrics synchronization. The code implementation of NTP time calibration has been given in the song synchronization, and the specific code implementation for the latter two parts will be provided below.
Key code implementation
1. Turn on black frame compensation
// In pure audio mode, the main instance (vocal instance) needs to turn on black frame compensation to carry the SEI message.
The frequency of the lead singer sending SEI messages is determined by the frequency of background music playback event callbacks, usually 200ms;
The reason for not directly using CMD messages to send song progress: The signaling transmitted by the SEI channel can be transmitted to the live CDN along with the video frames, providing better compatibility for the audience pulling the CDN stream.
long bgmProgressTime = jsonObject.getLong("bgmProgressTime");
long ntpTime = jsonObject.getLong("ntpTime");
String musicId = jsonObject.getString("musicId");
long duration = jsonObject.getLong("duration");
...
// TODO Update lyrics widget logic:
// If you reuse the TUIKaraoke component's lyrics widget,
//please refer to the code logic of the TUIKaraoke LyricsView section to synchronize the lyrics widget progress.
...
}
Note:
If you reuse the TUIKaraoke component's lyrics widget, please refer to the code logic of the TUIKaraoke LyricsView section to synchronize the lyrics widget progress.