Beauty AR SDK Core Interface Class XmagicApi.java, utilized for initializing the SDK, updating beauty metric values, invoking animated effects, amongst other features.
The overall calling process of each API is as follows:
List of static properties and methods of XmagicApi
API
Description
VERSION
The SDK version number can be retrieved via XmagicApi.VERSION (added in V3.5.0).
Transfer the content located within the directories Light3DPlugin, LightCore, LightHandPlugin, LightBodyPlugin, LightSegmentPlugin under application assets to your designated directory.
Set the SDK's log level, which defaults to Log.WARN. During development and debugging, it can be set to Log.DEBUG if necessary. For official release, make sure to set it to Log.WARN or Log.ERROR to avoid performance issues caused by excessive logging.
Invoke this method to enable the high-performance pattern. Upon the activation of the high-performance pattern, the system CPU/GPU resources occupied by beauty filters are minimized, thereby reducing heat generation and latency issues in the mobile device. It is particularly suitable for prolonged use on low-end devices.
Set the path of the so Library and trigger loading.
If the so Library is built into the apk package, this interface is not needed. If the so Library is downloaded dynamically, it should be invoked before authentication and new XmagicApi().
Passing in null indicates loading the so from the default path. Please ensure that the so is built into the APK package.
Pass in non-null: such as data/data/package name/files/xmagic_libs, the so will be loaded from this directory.
staticbooleansetLibPathAndLoad(String path)
Static method: addAiModeFilesFromAssets
SDK's Model Resource Files can be built into the assets directory of the apk package or downloaded dynamically. If they are built into the assets directory, the model resource files need to be copied to the specified directory before the SDK is used for the first time. This operation only needs to be done once. After upgrading the SDK version, it needs to be executed again.
context applicationcontext.
resDir is the root directory for storing SDK model resources. This directory is the same as the path parameter passed in when creating a new XmagicApi() object.
If you have not embedded the SDK's model resource file in the APK package but download it dynamically, after downloading and decompressing, you can call this method to copy the downloaded SDK model resource files to the corresponding folder.
inputResDir is the directory of the successfully downloaded model files.
resDir is the root directory for storing SDK model resources. This directory is the same as the path parameter passed in when creating a new XmagicApi() object.
During versions V3.7.0 - V3.9.0, if you need to call the getDeviceLevel method before new XmagicApi, you must include the benchmark folder from the SDK's assets directory into your APK's assets. If you call getDeviceLevel after new XmagicApi, there is no need to include it.
It is used to specify performance and effect modes, with possible values being NORMAL and PRO. NORMAL can meet most scenarios with better performance but fewer features. Please choose according to your needs.
Additionally, before instantiating XmagicApi, please first invoke the static method addAiModeFilesFromAssets or addAiModeFiles of XmagicApi to copy the model resource files to resDir.
xmagicPropertyErrorListener
OnXmagicPropertyErrorListener
Error Callback Interface. During the use of the SDK, this interface will callback some internal error information. Generally used during development and debugging.
OnXmagicPropertyErrorListener returns a table of error codes and their meanings:
Error code
Meaning
-1
Unknown error.
-100
3D engine resource initialization failed.
-200
GAN materials are not supported.
-300
This device does not support this material component.
-400
The template JSON content is empty.
-500
The SDK version is too low.
-600
Splitting is not supported.
-700
OpenGL is not supported.
-800
Scripting is not supported.
5,000
The resolution of the split background image exceeds 2160x3840.
5001
Insufficient memory required to segment the background image.
5002
Failed to parse the video segmentation of the background.
5003
Background video segment exceeds 200 seconds.
5004
Background video segment format unsupported.
5005
Background image segment possesses rotation angle
process
Methods for rendering data with the SDK, used for processing images or video streams. When processing video streams, it can be used within the camera data callback function.
The texture that needs to be rendered. The type is: OpenGL 2D texture format, and the pixel format is RGBA.
int srcTextureWidth
Width of the texture that needs to be rendered.
int srcTextureHeight
Height of the texture that needs to be rendered.
Bitmap bitmap
The recommended maximum size is 2160 x 4096. Larger images have poor face recognition results or cannot get faces recognized and are likely to cause OOM problems. Shrink such images first before passing them in.
boolean needReset
In the following scenarios, set needReset to true:
Processing an image for the first time, or switching images.
First time using partition.
Initial use of animated effects.
First-time usage of makeup.
setEffect
Set effects such as beauty, aesthetic shape, filters, makeup, stickers, and segmentation, and can be invoked in any thread. For specific parameters, see Effect Parameters.
Invoke after new XmagicApi(). Set the SDK's log level, which defaults to Log.WARN. During development and debugging, it can be set to Log.DEBUG if necessary. For official release, make sure to set it to Log.WARN or Log.ERROR to avoid performance issues caused by excessive logging.
If ITELogger is set, the SDK's internal logs will be redirected to the user.
Invoke this method to enable the High-Performance Mode, new in V3.7.0. After enabling the High-Performance Mode, the system CPU/GPU resources used for beauty filters are minimized, reducing heat generation and latency issues on the mobile device. This mode is more suitable for prolonged use on low-end devices. However, compared to the normal mode, some features may be limited.
It needs to be invoked immediately after new XmagicApi(). For detailed instructions, refer to the High-Performance Mode Usage Guide.
voidenableHighPerformance()
setFeatureEnableDisable
Enable or disable a specific capability as needed.
XmagicConstant.FeatureName.SEGMENTATION_SKIN Skin Segmentation Capability, after enabling, can make the skin smoothing and whitening areas more accurate.
XmagicConstant.FeatureName.SEGMENTATION_FACE_BLOCK Face Occlusion Detection Capability, after enabling, can avoid makeup being applied to occluded areas.
XmagicConstant.FeatureName.WHITEN_ONLY_SKIN_AREA Whitening only works on skin
XmagicConstant.FeatureName.SMART_BEAUTY Intelligent Beauty (reduces beauty and makeup effects for men and babies)
XmagicConstant.FeatureName.ANIMOJI_52_EXPRESSION Face Expression Capability
XmagicConstant.FeatureName.BODY_3D_POINT Body Point Capability
true indicates to enable this capability, false indicates to disable this capability
Among the mentioned capabilities, the commonly used ones are SEGMENTATION_SKIN and SEGMENTATION_FACE_BLOCK. The SDK will enable these two capabilities by default based on the value of getDeviceLevel. If you need to enable them manually, it is recommended to enable SEGMENTATION_SKIN when level >= 4, and enable SEGMENTATION_FACE_BLOCK when level >= 5.
setXmagicStreamType
Set the input data type; there are two types: camera data and image data. The default is camera data stream.
voidsetXmagicStreamType(int type)
Parameter
Meaning
int type
Data source type, there are two options:
XmagicApi.PROCESS_TYPE_CAMERA_STREAM: camera data source.
XmagicApi.PROCESS_TYPE_PICTURE_DATA: Image data source.
The SDK will callback after starting to process the image. When a face is recognized, it callbacks List<FaceData>, the size of the list is the number of faces. When there are no faces, it callbacks an empty List.
The definition of TEFaceData is as follows: points are coordinates for 83 face points, each with x and y coordinates, so the length of points is fixed at 166. The coordinate values are not based on the original input image but are face coordinates obtained after scaling the short edge of the original image to 256.
Therefore, it is recommended to use the callback data here only to determine whether there are faces in the current screen and how many faces there are. If you need higher precision face points, please use the data from onAIDataUpdated callback.
publicclassTEFaceData{
publicfloat[] points;
publicTEFaceData(){
}
publicTEFaceData(float[] points){
this.points = points;
}
}
onHandDataUpdated
Deprecated interface, no data callback. The old version SDK callbacks when gesture effects are set and gestures are recognized.
If gesture data is needed, please use the data from onAIDataUpdated callback.
onBodyDataUpdated
Callbacks when body attributes are set and the body is recognized; no callbacks in other cases. The callback is the coordinates of the body's 42 points, the coordinate values are based on the width and height of the input image. If a point is not detected, the coordinate value is 0.
onAIDataUpdated
Callback detailed data of face, gesture, and body 3D usually requires a specific license to have data. An example data is as follows:
Set the image orientation so that AI can recognize faces from different directions. If set, the direction provided by sensorChanged will be ignored. Example directions are as follows:
If the image you provide to the SDK is always upright (Rotation = 0), then you do not need to invoke this interface.
Use the system sensor to determine the current phone's rotation angle, so that the AI can recognize faces in different orientations. Note: If a fixed direction is set through setImageOrientation, the direction provided by sensorChanged will be ignored.
Pass the path of a dynamic effect material to the SDK, and detect whether the current device fully supports this dynamic effect.
For example, if a dynamic effect contains lipstick, facial stickers, and background segmentation, if the current device does not support background segmentation, this interface will return false. False does not mean that the dynamic effect cannot be used at all, but some effects cannot be presented.
isSupportBeauty
Determine whether the current model supports beauty (OpenGL3.0). Currently, most mobile phones on the market support OpenGL3.0, so usually there is no need to use this interface.
Establish the callback for animated effect cues, utilized for displaying prompts onto the frontend page. For instance, certain materials might instruct users to nod their heads, extend their palms, or make a heart shape.
Note: The callback method may not be on the main thread.
XmagicTipsListener includes the following methods:
publicinterfaceXmagicTipsListener{
/**
* Display tips.
* @param tips The returned prompt text information.
* @param tipsIcon The file path of the tips icon. You can parse the corresponding image based on the file path. If it's a pag file, you need to use pagView to display it.
* Note: tipsIcon may be empty, return null if not configured in the material.
* @param type Tips category, 0 means both tips and tipsIcon have values, 1 means it is a pag material and only tipsIcon has values.
* @param duration Tips display duration, in milliseconds.
* @param tips The returned prompt text information.
* @param tipsIcon The file path of the tips icon. You can parse the corresponding image based on the file path. If it's a pag file, you need to use pagView to display it.
* Note: tipsIcon may be empty, return null if not configured in the material.
* @param type Tips category, 0 means both tips and tipsIcon have values, 1 means it is a pag material and only tipsIcon has values.
Some recognition and rendering logic inside the SDK is processed asynchronously. By calling this interface, you can make the SDK process the input images synchronously for the next syncFrameCount frames to meet specific requirements in certain scenarios. For example, before processing the first frame, calling this interface allows the SDK to process several frames synchronously, which can prevent unfiltered images from being displayed. However, this may increase the black screen duration before the first frame is rendered, so please use it as needed.