接口文档
Beauty AR SDK Core Interface Class
XmagicApi.java
, utilized for initializing the SDK, updating beauty metric values, invoking animated effects, amongst other features.The overall calling process of each API is as follows:
Public Member Functions
API | Description |
constructor. | |
Establishing beauty enhancements, styling, filters, makeup applications, stickers, division and other effects can be invoked in any thread (New in V3.5.0 version) | |
Setting up callback functions for animated hint text, designated to display hints on the frontend page. | |
Configure the callback of facial keypoints and other data, callback will only be available with the License authorization required to acquire facial keypoints. | |
Configure the callback for face, gesture, and body detection statuses. | |
Pause audio playback, which can be associated with the `onPause` lifetime affinity of `Activity`. | |
Resume rendering, can be paired with Activity onResume lifetime affinity. | |
Terminate `xmagic`, which necessitates its invocation within the `GL` thread. | |
The method for SDK rendering to accept data can be used within the camera data callback function. | |
Invoke this function when only the cessation of the audio is necessitated, without the need to release the GL thread. | |
It serves to establish the present angle of rotation of the mobile phone, thus adjusting the angle for AI to recognize faces based on this. | |
The list of dynamic effect resources is introduced into the SDK for examination, executing afterwards the XmagicProperty.isSupport field represents whether the atomic capability is applicable. According to XmagicProperty.isSupport , the UI layer can implement click restrictions, or directly eliminate from the resource list. | |
Pass in a list of dynamic effect resources and return the list of SDK atomic capabilities used by each resource. | |
Return the atomic capability list supported by the current device. | |
Determine whether the current device supports refinement (OpenGL3.0). | |
Determine which beauties the current lic authorization supports. It only supports the detection of BEAUTY and BODY_BEAUTY types of beauty items. The detection result will be assigned to the XmagicProperty.isAuth field of each beauty object. | |
Set the input data type, the default is Android camera data stream. | |
Establish the log level of the SDK, the default is WARN . During the development and debugging stage, if needed, it can be set to Log.DEBUG. Be sure to set to Log.WARN or Log.ERROR upon official release, otherwise excessive logs will affect performance.Invoke after new XmagicApi(). | |
Whether to mute when using animation materials (Added in V2.5.0): Parameter: true indicates mute, false denotes non-mute. | |
Activate the beautification enhancing pattern (Added in V2.5.1). By default, it is not activated. When it's disabled, the application layer can set the strength range for each beauty field between 0 to 1 or -1 to 1. If it exceeds the boundary, SDK will fixate to the boundary value. For example, if the application layer sets face-slimming component to 1.2, SDK will limit it to the maximum value of 1.0. After enabling the enhanced pattern, the application layer can set a broader range of values. For instance, if a greater face slimming effect is desired, you can set the slimming value to 1.2. The SDK will accept and use this value of 1.2 without correcting it to 1.0. Note: Upon enabling the enhancement pattern, the application layer must manage the maximum settable value for each beauty field, allowing users to adjust the value within such boundary parameters. We offer a reference value that you can adjust in accordance to product demand. However, we do not recommend exceeding our advised estimates as this could potentially deteriorate the beauty effect. | |
| |
Invoke this method to enable the high-performance pattern. Upon the activation of the high-performance pattern, the system CPU/GPU resources occupied by beauty filters are minimized, thereby reducing heat generation and latency issues in the mobile device. It is particularly suitable for prolonged use on low-end devices. | |
Get the picture on the current texture | |
Enable or disable a certain capability. |
XmagicApi
constructor.
XmagicApi(Context context, String resDir)XmagicApi(Context context, String resDir,OnXmagicPropertyErrorListener xmagicPropertyErrorListener)
Parameter
Parameter | Type | Meaning |
context | Context | context. |
resDir | String | Resource directory. If the SDK resources are built into assets, before the first use of the SDK, the resources need to be copied to the application's private directory. The resource path is set first via XmagicResParser.setResPath(new File(getFilesDir(), "xmagic").getAbsolutePath()) , and then the resource copying is completed via XmagicResParser.copyRes(getApplicationContext()) . See the TEMenuActivity.java document in the Demo for detailed information.If the SDK resources are downloaded from the internet, after a successful download, the resource path can be set by XmagicResParser.setResPath(validAssetsDirectory) .Access the previously set path via XmagicResParser.getResPath() . |
xmagicPropertyErrorListener | OnXmagicPropertyErrorListener | Error callback interface. |
Return the reference table for the meanings of error codes:
Error code | Meaning |
-1 | Unknown error. |
-100 | 3D engine resource initialization failed. |
-200 | GAN materials are not supported. |
-300 | This device does not support this material component. |
-400 | The template JSON content is empty. |
-500 | The SDK version is too low. |
-600 | Splitting is not supported. |
-700 | OpenGL is not supported. |
-800 | Scripting is not supported. |
5000 | The resolution of the split background image exceeds 2160x3840. |
5001 | Insufficient memory required to segment the background image. |
5002 | Failed to parse the video segmentation of the background. |
5003 | Background video segment exceeds 200 seconds. |
5004 | Background video segment format unsupported. |
5005 | Background image segment possesses rotation angle |
setEffect(Added in V3.5.0)
You can configure beautification, shaping, filtering, makeup, stickers, partitioning, and other effects on any thread. For specific parameters, please refer to Effect Parameters.
void setEffect(String effectName, int effectValue, String resourcePath, Map<String, String> extraInfo)
setTipsListener
Establish the callback for animated effect cues, utilized for displaying prompts onto the frontend page. For instance, certain materials might instruct users to nod their heads, extend their palms, or make a heart shape.
void setTipsListener(XmagicApi.XmagicTipsListener effectTipsListener)
Parameter
Parameter | Meaning |
XmagicApi.XmagicTipsListener effectTipsListener | Implementation class of the callback function, callbacks are not necessarily executed in the main thread. |
setYTDataListener(this interface was removed in version 3.0.0, the feature has been migrated to the onAIDataUpdated method in XmagicAIDataListener)
Configure callbacks for facial landmark information and other data.
void setYTDataListener(XmagicApi.XmagicYTDataListener ytDataListener)Configuring the callback for face information and other datapublic interface XmagicYTDataListener {void onYTDataUpdate(String data)}
onYTDataUpdate returns a JSON string structure, providing a maximum of five facial information:
{"face_info":[{"trace_id":5,"face_256_point":[180.0,112.2,...],"face_256_visible":[0.85,...],"out_of_screen":true,"left_eye_high_vis_ratio:1.0,"right_eye_high_vis_ratio":1.0,"left_eyebrow_high_vis_ratio":1.0,"right_eyebrow_high_vis_ratio":1.0,"mouth_high_vis_ratio":1.0},...]}
Field Description
Field | Type | Range | Description |
trace_id | int | [1,INF) | Face ID. The same ID points to the same face in the process of continuous stream fetching. |
face_256_point | float | [0,screenWidth] or [0,screenHeight] | 512 values in total for 256 facial keypoints. (0,0) is the top-left corner of the screen. |
face_256_visible | float | [0,1] | Visibility of the 256 facial keypoints. |
out_of_screen | bool | true/false | Whether the face is out of the screen. |
left_eye_high_vis_ratio | float | [0,1] | Percentage of the highly visible points of the left eye. |
right_eye_high_vis_ratio | float | [0,1] | Percentage of the highly visible points of the right eye. |
left_eyebrow_high_vis_ratio | float | [0,1] | Percentage of the highly visible points of the left eyebrow. |
right_eyebrow_high_vis_ratio | float | [0,1] | Percentage of the highly visible points of the right eyebrow. |
mouth_high_vis_ratio | float | [0,1] | Percentage of the highly visible points of the mouth. |
Parameter
Parameter | Meaning |
XmagicApi.XmagicYTDataListener ytDataListener | Callback function implementation class. |
setAIDataListener
Upon detection of faces, bodies, and gestures, the spatial information of these parts will be returned via a callback:
onFaceDataUpdated: Once beauty mode is enabled, there will be a callback. When a face is detected, a List<FaceData> is returned. If no face is detected, an empty List is returned.
onHandDataUpdated: A callback is established when gestures are recognized after setting the gesture animation. No callbacks occur under any other circumstances.
onBodyDataUpdated: Callback when the body shaping properties are set and the body is recognized. It does not callback in other situations.
public interface OnAIDataListener {void onFaceDataUpdated(List<TEFaceData> faceDataList);void onHandDataUpdated(List<TEHandData> handDataList);void onBodyDataUpdated(List<TEBodyData> bodyDataList);void onAIDataUpdated(String jsonString); // This method is a new addition in version 3.0.0, and the data structure aligns with the preceding XmagicYTDataListener interface's data}
onPause
Pause rendering, which can be bound with Activity's onPause lifetime affinity. Currently, only
onPauseAudio
is called internally.void onPause()
onResume
Resume rendering, can be paired with Activity onResume lifetime affinity.
void onResume()
onDestroy
Clears GL thread resources and needs to be called within the GL thread. Sample code:
// Refer to the sample code in `TECameraBaseActivity.java`public void onGLContextDestroy() { if (this.mXMagicApi != null) { this.mXMagicApi.onDestroy(); this.mXMagicApi = null; } }
process
The method for SDK rendering to accept data can be used within the camera data callback function.
// Render the textureint process(int srcTextureId, int srcTextureWidth, int srcTextureHeight)// Render the bitmapBitmap process(Bitmap bitmap, boolean needReset)
Parameter
Parameter | Meaning |
int srcTextureId | The texture that needs to be rendered. The type is: OpenGL 2D texture format, and the pixel format is RGBA. |
id int srcTextureWidth | Width of the texture that needs to be rendered. |
int srcTextureHeight | Height of the texture that needs to be rendered. |
Bitmap bitmap | The recommended maximum size is 2160 x 4096. Larger images have poor face recognition results or cannot get faces recognized and are likely to cause OOM problems. Shrink such images first before passing them in. |
boolean needReset | Switch the image. First time using partition. Initial use of animated effects. First-time usage of makeup. For these scenarios, the needReset Setting should be set to `true`. |
onPauseAudio
Invoke this function when only the cessation of the audio is necessitated, without the need to release the GL thread.
void onPauseAudio()
sensorChanged
Used to determine the current rotation angle of the phone, so as to adjust the basis for AI to recognize the angle of the face. This needs to be called in the callback function of the G-sensor.
void sensorChanged(SensorEvent event, Sensor accelerometer)
Parameter
Parameter | Meaning |
SensorEvent event | Event entity class returned by the gyroscope sensor callback function onSensorChanged . |
Sensor accelerometer | Sample G-sensor. |
isDeviceSupport
version 3.5.0 and later
/*** Check whether the current device supports this material* @param motionResPath The path of the material file* @return true means supported, false means not supported*/boolean isDeviceSupport(String motionResPath)
version 3.3.0 and earlier
Transmit the list of animated effect resources into the SDK for verification. After execution, the
XmagicProperty.isSupport
field signifies if the resource is serviceable. Based on XmagicProperty.isSupport
, click control can be enacted at the UI level, or it can be directly expunged from the resource list.void isDeviceSupport(List<XmagicProperty<?>> assetsList)
Parameter
Parameter | Meaning |
List<XmagicProperty<?>> assetsList | List of animated effect materials to be checked. |
getPropertyRequiredAbilities
Inputs an animated effect resources list, returns the SDK atomic abilities list used for each resource.
The usage scenario of this method:
If you have purchased or created a number of animated effect materials, by invoking this method, it will returning a list of atomic abilities required for each material. For instance, material 1 requires abilities A, B, C while material 2 need abilities B, C, D. Subsequently, you maintain such a list of abilities on your server. Later on, when a user wants to download the animated effect materials from the server, the user first accesses the list of atomic abilities that his mobile has through the getDeviceAbilities method (for example, this phone possess abilities A, B, C, but lacks D), transmits this abilities list to the server. The server, on determining the device does not have ability D, therefore does not issue material 2 to the user.
Parameter
Parameter | Meaning |
List<XmagicProperty<?>> assets | List of animated effect resources for which to check the atomic capabilities. |
Return
Returned value
Map<XmagicProperty<?>,ArrayList<String>>
:key: animated effect resource material entity class.
value: list of used atomic capabilities.
getDeviceAbilities
It returns the atomic capability table that the current device supports. To be used in conjunction with the getPropertyRequiredAbilities method. For more details, please refer to the description of the getPropertyRequiredAbilities.
Map<String,Boolean> getDeviceAbilities()
Return
Return value
Map<String,Boolean>
:key: atomic capability name (corresponding to the material capability name).
value: whether it is supported by the current device.
isSupportBeauty
Determine whether the current device supports refinement (OpenGL3.0).
boolean isSupportBeauty()
Return
Return value boolean: Determines whether the beauty filter is supported.
isBeautyAuthorized
Determine which beauty or body modifier features the current License authorization supports. Only detection of BEAUTY and BODY_BEAUTY types are supported. The resultant evaluation will be allocated to each beauty object's
XmagicProperty.isAuth
field. If the isAuth field returns false, the entrances for these features can be concealed in the UI.void isBeautyAuthorized(List<XmagicProperty<?>> properties)
Parameter
Parameter | Meaning |
List<XmagicProperty<?>> properties | Beauty filters that need to be checked. |
setXmagicStreamType
Sets the input data type, which is Android camera data stream (XmagicApi.PROCESS_TYPE_CAMERA_STREAM) by default.
void setXmagicStreamType(int type)
Parameter
Parameter | Meaning |
int type | Data source type, there are two options: XmagicApi.PROCESS_TYPE_CAMERA_STREAM : camera data source.XmagicApi.PROCESS_TYPE_PICTURE_DATA : Image data source. |
setXmagicLogLevel
Setting the log level of the SDK, the default is WARN. During the development debugging stage if necessary, it can be set to
Log.DEBUG.
. When officially released, it must be set to Log.WARN or Log.ERROR
otherwise, an immense amount of logs will adversely affect performance.Invoke after new XmagicApi().
setAudioMute
Whether to mute when using animation materials (Added in V2.5.0):
Parameter: true indicates mute, false denotes non-mute.
enableEnhancedMode
Activate the beautification enhancing pattern (Added in V2.5.1). By default, it is not activated.
When inactive, the strength range of each beauty option in the application layer can be set from 0 to 1 or -1 to 1. If this range is exceeded, the SDK will take the boundary value. For instance, if the application layer sets the face-thinning to 1.2, the SDK determines that this surpasses the maximum value of 1.0, and internally adjusts the face-thinning value to 1.0.
After enabling the enhanced pattern, the application layer can set a broader range of values. For instance, if a greater face slimming effect is desired, you can set the slimming value to 1.2. The SDK will accept and use this value of 1.2 without correcting it to 1.0.
Upon activation of the enhanced pattern, it becomes the responsibility of the application layer to manage the maximal values of each beauty setting, allowing users to adjust within this range. We provide a set of reference values which you may adjust freely according to product requirements, though we advise not exceeding our recommended values to avoid potential degradation of beauty effects. The reference values are as follows:
Beauty Item Name | In enhanced pattern, recommended maximum value (magnification factor) |
Whitening, shortening the face, V-face, eye distance, nose position, removal of laugh lines, lipstick, three-dimensional appearance | 1.3 |
Eye lightening | 1.5 |
Blush | 1.8 |
Other | 1.2 |
| |
| |
| |
enableHighPerformance
Upon activating the advanced performance pattern, the resources consumed by the beautification function on your system's CPU/GPU are diminished, effectively mitigating the heating and lagging issues of your mobile device, making it particularly beneficial for extended utilization on lower-end devices.
Note: After enabling the high performance pattern, the following beauty options will be unavailable:
1. Ocular region: Eye width, eye height, removal of eye bags .
2. Eyebrows: Angle, Distance, Height, Length, Thickness, Eyebrow Peak.
3. Mouth: The Smiling Lips.
4. Facial: Slimming (natural, goddess, handsome), jaw reduction, wrinkle removal, and nasolabial fold removal. It is recommended to use "face shape" to achieve the comprehensive effect of large eyes and slim face.
void boolean enableHighPerformance()
exportCurrentTexture
Get the picture on the current texture
void exportCurrentTexture(ExportTextureCallback callback)
setFeatureEnableDisable
Enable or disable a feature.
void setFeatureEnableDisable(String featureName, boolean enable)
Parameter
Parameter | Meaning |
String featureName | feature Name Values: XmagicConstant.FeatureName.ANIMOJI_52_EXPRESSION facial expressions feature.XmagicConstant.FeatureName.BODY_3D_POINT 3D body data feature.XmagicConstant.FeatureName.HAND_DETECT gesture detection.XmagicConstant.FeatureName.WHITEN_ONLY_SKIN_AREA Brightening only applies to skin.XmagicConstant.FeatureName.SEGMENTATION_SKIN segmentation skin.XmagicConstant.FeatureName.SMART_BEAUTY smart beauty(reducing the intensity of beauty and makeup effects for males and babies). |
boolean enable | "true" indicates enabling a capability, while "false" indicates disabling a capability. Note: If it is in downgrade mode, enabling skin segmentation is not allowed. |
Static properties and methods
API | Description |
VERSION | The SDK version number can be obtained through XmagicApi.VERSION (Added in V3.5.0) |
Setting libPath. | |
Transfer the content located within the directories Light3DPlugin, LightCore, LightHandPlugin, LightBodyPlugin, LightSegmentPlugin under application assets to your designated directory. | |
Copy the AI model files downloaded by the client to the corresponding folders. |
setLibPathAndLoad
Setting the path for SO and triggering a load. If SO is built into the assets, there is no need to invoke this method. If SO is dynamically downloaded, it must be invoked before authentication and
new XmagicApi
. Passing in null indicates loading the so from the default path. Please ensure that the so is built into the APK package.
Pass in non-null: such as
data/data/package name/files/xmagic_libs
, the so will be loaded from this directory.static boolean setLibPathAndLoad(String path)
Parameter
Parameter | Meaning |
String path | so: denotes the path to the library. |
addAiModeFilesFromAssets
Transfer the content located within the directories Light3DPlugin, LightCore, LightHandPlugin, LightBodyPlugin, LightSegmentPlugin under application assets to your designated directory.
context applicationcontext.
resDir serves as the root directory for storing beautification resources. This directory corresponds to the path passed when creating an xmagicApi object.
Returned values:
0: Copy successful
-1: denotes that the context is null
-2: denotes an IO error
static int addAiModeFilesFromAssets(Context context, String resDir)
Parameter
Parameter | Meaning |
Context context | applicationcontext. |
String resDir | Designated as the root directory for storing beautification resources, this directory corresponds to the path input when creating the xmagicApi object. |
addAiModeFiles
Copy the AI model files downloaded by the client to the corresponding folders.
inputResDir is the directory of the successfully downloaded model files.
resDir serves as the root directory for storing beautification resources. This directory corresponds to the path passed when creating an xmagicApi object.
Returned values:
0: denotes success
-1:inputResDir is not exists
-2: Signifies an IO error
static int addAiModeFiles(String inputResDir, String resDir)
Parameter
Parameter | Meaning |
String inputResDir | The folder containing the successfully downloaded model files. |
String resDir | Designated as the root directory for storing beautification resources, this directory corresponds to the path input when creating the xmagicApi object. |
- Public Member Functions
- XmagicApi
- setEffect(Added in V3.5.0)
- setTipsListener
- setYTDataListener(this interface was removed in version 3.0.0, the feature has been migrated to the onAIDataUpdated method in XmagicAIDataListener)
- setAIDataListener
- onPause
- onResume
- onDestroy
- process
- onPauseAudio
- sensorChanged
- isDeviceSupport
- getPropertyRequiredAbilities
- getDeviceAbilities
- isSupportBeauty
- isBeautyAuthorized
- setXmagicStreamType
- setXmagicLogLevel
- setAudioMute
- enableEnhancedMode
- enableHighPerformance
- exportCurrentTexture
- setFeatureEnableDisable
- Static properties and methods