Beauty AR
  • Overview
  • Web
    • クイックスタート
    • インテグレーション
      • Overview
      • 内蔵カメラ
      • カスタムストリーム
      • 読み込みの最適化
      • エフェクト設定
      • 人物画像分割の使用
      • 絵文字とバーチャルキャラクターの設定
    • インターフェースドキュメンテーション
    • リリース説明
    • ベストプラクティス
      • WebRTC によるプッシュ
      • WebRTC によるプッシュ(事前初期化スキーム)
      • TRTC によるプッシュ
    • よくあるご質問
  • Android
    • インテグレーション
      • SDK の統合
      • TEBeautyKit の統合
    • インターフェースドキュメンテーション
    • リリース説明
    • ベストプラクティス
      • SDK サイズの削減
      • エフェクトパラメータ
    • 高度な機能
      • ジェスチャー認識
      • 顔認識
      • 仮想背景
    • 素材制作
      • 美顔 AR Studio の紹介
    • よくあるご質問
  • IOS
    • インテグレーション
      • SDK の統合
      • TEBeautyKit の統合
    • インターフェースドキュメンテーション
    • リリース説明
    • ベストプラクティス
      • SDK サイズの削減
      • エフェクトパラメータ
    • 高度な機能
      • ジェスチャー認識
      • 顔認識
      • 仮想背景
    • 素材制作
      • 美顔 AR Studio の紹介
    • よくあるご質問
  • Flutter
    • インテグレーション
    • インターフェースドキュメンテーション
    • 素材制作
      • 美顔 AR Studio の紹介
  • Overview
    • Overview
  • サービスをアクティブ化します
  • 価格
  • 体験バージョン
    • Web
    • モバイル端末
Beauty AR
このページは現在英語版のみで提供されており、日本語版も近日中に提供される予定です。ご利用いただきありがとうございます。

インターフェースドキュメンテーション

Beauty AR SDK Core Interface Class XmagicApi.java, utilized for initializing the SDK, updating beauty metric values, invoking animated effects, amongst other features.

Public Member Functions

API
Description
XmagicApi
constructor.
setEffect
Establishing beauty enhancements, styling, filters, makeup applications, stickers, division and other effects can be invoked in any thread (New in V3.5.0 version)
Setting up callback functions for animated hint text, designated to display hints on the frontend page.
Configure the callback of facial keypoints and other data, callback will only be available with the License authorization required to acquire facial keypoints.
Configure the callback for face, gesture, and body detection statuses.
onPause
Pause audio playback, which can be associated with the `onPause` lifetime affinity of `Activity`.
onResume
Resume rendering, can be paired with Activity onResume lifetime affinity.
onDestroy
Terminate `xmagic`, which necessitates its invocation within the `GL` thread.
process
The method for SDK rendering to accept data can be used within the camera data callback function.
Invoke this function when only the cessation of the audio is necessitated, without the need to release the GL thread.
It serves to establish the present angle of rotation of the mobile phone, thus adjusting the angle for AI to recognize faces based on this.
The list of dynamic effect resources is introduced into the SDK for examination, executing afterwards the XmagicProperty.isSupport field represents whether the atomic capability is applicable. According to XmagicProperty.isSupport, the UI layer can implement click restrictions, or directly eliminate from the resource list.
Pass in a list of dynamic effect resources and return the list of SDK atomic capabilities used by each resource.
Return the atomic capability list supported by the current device.
Determine whether the current device supports refinement (OpenGL3.0).
Determine which beauties the current lic authorization supports. It only supports the detection of BEAUTY and BODY_BEAUTY types of beauty items. The detection result will be assigned to the XmagicProperty.isAuth field of each beauty object.
Set the input data type, the default is Android camera data stream.
Establish the log level of the SDK, the default is WARN. During the development and debugging stage, if needed, it can be set to Log.DEBUG. Be sure to set to Log.WARN or Log.ERROR upon official release, otherwise excessive logs will affect performance.
Invoke after new XmagicApi().
Whether to mute when using animation materials (Added in V2.5.0):
Parameter: true indicates mute, false denotes non-mute.
Activate the beautification enhancing pattern (Added in V2.5.1). By default, it is not activated.
When it's disabled, the application layer can set the strength range for each beauty field between 0 to 1 or -1 to 1. If it exceeds the boundary, SDK will fixate to the boundary value. For example, if the application layer sets face-slimming component to 1.2, SDK will limit it to the maximum value of 1.0.
After enabling the enhanced pattern, the application layer can set a broader range of values. For instance, if a greater face slimming effect is desired, you can set the slimming value to 1.2. The SDK will accept and use this value of 1.2 without correcting it to 1.0.
Note:
Upon enabling the enhancement pattern, the application layer must manage the maximum settable value for each beauty field, allowing users to adjust the value within such boundary parameters. We offer a reference value that you can adjust in accordance to product demand. However, we do not recommend exceeding our advised estimates as this could potentially deteriorate the beauty effect.
Invoke this method to enable the high-performance pattern. Upon the activation of the high-performance pattern, the system CPU/GPU resources occupied by beauty filters are minimized, thereby reducing heat generation and latency issues in the mobile device. It is particularly suitable for prolonged use on low-end devices.
Get the picture on the current texture
Enable or disable a certain capability.

XmagicApi

constructor.
XmagicApi(Context context, String resDir)
XmagicApi(Context context, String resDir,OnXmagicPropertyErrorListener xmagicPropertyErrorListener)

Parameter

Parameter
Type
Meaning
context
Context
context.
resDir
String
Resource directory.
If the SDK resources are built into assets, before the first use of the SDK, the resources need to be copied to the application's private directory. The resource path is set first via XmagicResParser.setResPath(new File(getFilesDir(), "xmagic").getAbsolutePath()), and then the resource copying is completed via XmagicResParser.copyRes(getApplicationContext()). See the TEMenuActivity.java document in the Demo for detailed information.
If the SDK resources are downloaded from the internet, after a successful download, the resource path can be set by XmagicResParser.setResPath(validAssetsDirectory).
Access the previously set path via XmagicResParser.getResPath().
xmagicPropertyErrorListener
OnXmagicPropertyErrorListener
Error callback interface.
Return the reference table for the meanings of error codes:
Error code
Meaning
-1
Unknown error.
-100
3D engine resource initialization failed.
-200
GAN materials are not supported.
-300
This device does not support this material component.
-400
The template JSON content is empty.
-500
The SDK version is too low.
-600
Splitting is not supported.
-700
OpenGL is not supported.
-800
Scripting is not supported.
5000
The resolution of the split background image exceeds 2160x3840.
5001
Insufficient memory required to segment the background image.
5002
Failed to parse the video segmentation of the background.
5003
Background video segment exceeds 200 seconds.
5004
Background video segment format unsupported.
5005
Background image segment possesses rotation angle


setEffect(Added in V3.5.0)

You can configure beautification, shaping, filtering, makeup, stickers, partitioning, and other effects on any thread. For specific parameters, please refer to Effect Parameters.
void setEffect(String effectName, int effectValue, String resourcePath, Map<String, String> extraInfo)

setTipsListener

Establish the callback for animated effect cues, utilized for displaying prompts onto the frontend page. For instance, certain materials might instruct users to nod their heads, extend their palms, or make a heart shape.
void setTipsListener(XmagicApi.XmagicTipsListener effectTipsListener)

Parameter

Parameter
Meaning
XmagicApi.XmagicTipsListener effectTipsListener
Implementation class of the callback function, callbacks are not necessarily executed in the main thread.

setYTDataListener(this interface was removed in version 3.0.0, the feature has been migrated to the onAIDataUpdated method in XmagicAIDataListener)

Configure callbacks for facial landmark information and other data.
void setYTDataListener(XmagicApi.XmagicYTDataListener ytDataListener)
Configuring the callback for face information and other data

public interface XmagicYTDataListener {
void onYTDataUpdate(String data)
}
onYTDataUpdate returns a JSON string structure, providing a maximum of five facial information:
{
"face_info":[{
"trace_id":5,
"face_256_point":[
180.0,
112.2,
...
],
"face_256_visible":[
0.85,
...
],
"out_of_screen":true,
"left_eye_high_vis_ratio:1.0,
"right_eye_high_vis_ratio":1.0,
"left_eyebrow_high_vis_ratio":1.0,
"right_eyebrow_high_vis_ratio":1.0,
"mouth_high_vis_ratio":1.0
},
...
]
}

Field Description

Field
Type
Range
Description
trace_id
int
[1,INF)
Face ID. The same ID points to the same face in the process of continuous stream fetching.
face_256_point
float
[0,screenWidth] or [0,screenHeight]
512 values in total for 256 facial keypoints. (0,0) is the top-left corner of the screen.
face_256_visible
float
[0,1]
Visibility of the 256 facial keypoints.
out_of_screen
bool
true/false
Whether the face is out of the screen.
left_eye_high_vis_ratio
float
[0,1]
Percentage of the highly visible points of the left eye.
right_eye_high_vis_ratio
float
[0,1]
Percentage of the highly visible points of the right eye.
left_eyebrow_high_vis_ratio
float
[0,1]
Percentage of the highly visible points of the left eyebrow.
right_eyebrow_high_vis_ratio
float
[0,1]
Percentage of the highly visible points of the right eyebrow.
mouth_high_vis_ratio
float
[0,1]
Percentage of the highly visible points of the mouth.

Parameter

Parameter
Meaning
XmagicApi.XmagicYTDataListener ytDataListener
Callback function implementation class.

setAIDataListener

Upon detection of faces, bodies, and gestures, the spatial information of these parts will be returned via a callback:
onFaceDataUpdated: Once beauty mode is enabled, there will be a callback. When a face is detected, a List<FaceData> is returned. If no face is detected, an empty List is returned.
onHandDataUpdated: A callback is established when gestures are recognized after setting the gesture animation. No callbacks occur under any other circumstances.
onBodyDataUpdated: Callback when the body shaping properties are set and the body is recognized. It does not callback in other situations.
public interface OnAIDataListener {

void onFaceDataUpdated(List<TEFaceData> faceDataList);
void onHandDataUpdated(List<TEHandData> handDataList);
void onBodyDataUpdated(List<TEBodyData> bodyDataList);
void onAIDataUpdated(String jsonString); // This method is a new addition in version 3.0.0, and the data structure aligns with the preceding XmagicYTDataListener interface's data
}

onPause

Pause rendering, which can be bound with Activity's onPause lifetime affinity. Currently, only onPauseAudio is called internally.
void onPause()

onResume

Resume rendering, can be paired with Activity onResume lifetime affinity.
void onResume()

onDestroy

Clears GL thread resources and needs to be called within the GL thread. Sample code:
// Refer to the sample code in `TECameraBaseActivity.java`
public void onGLContextDestroy() { if (this.mXMagicApi != null) { this.mXMagicApi.onDestroy(); this.mXMagicApi = null; } }

process

The method for SDK rendering to accept data can be used within the camera data callback function.
// Render the texture
int process(int srcTextureId, int srcTextureWidth, int srcTextureHeight)
// Render the bitmap
Bitmap process(Bitmap bitmap, boolean needReset)

Parameter

Parameter
Meaning
int srcTextureId
The texture that needs to be rendered. The type is: OpenGL 2D texture format, and the pixel format is RGBA.
id int srcTextureWidth
Width of the texture that needs to be rendered.
int srcTextureHeight
Height of the texture that needs to be rendered.
Bitmap bitmap
The recommended maximum size is 2160 x 4096. Larger images have poor face recognition results or cannot get faces recognized and are likely to cause OOM problems. Shrink such images first before passing them in.
boolean needReset
Switch the image.
First time using partition.
Initial use of animated effects.
First-time usage of makeup.
For these scenarios, the needReset Setting should be set to `true`.

onPauseAudio

Invoke this function when only the cessation of the audio is necessitated, without the need to release the GL thread.
void onPauseAudio()

sensorChanged

Used to determine the current rotation angle of the phone, so as to adjust the basis for AI to recognize the angle of the face. This needs to be called in the callback function of the G-sensor.
void sensorChanged(SensorEvent event, Sensor accelerometer)

Parameter

Parameter
Meaning
SensorEvent event
Event entity class returned by the gyroscope sensor callback function onSensorChanged.
Sensor accelerometer
Sample G-sensor.

isDeviceSupport

version 3.5.0 and later
/**
* Check whether the current device supports this material
* @param motionResPath The path of the material file
* @return true means supported, false means not supported
*/
boolean isDeviceSupport(String motionResPath)
version 3.3.0 and earlier
Transmit the list of animated effect resources into the SDK for verification. After execution, the XmagicProperty.isSupport field signifies if the resource is serviceable. Based on XmagicProperty.isSupport, click control can be enacted at the UI level, or it can be directly expunged from the resource list.
void isDeviceSupport(List<XmagicProperty<?>> assetsList)

Parameter

Parameter
Meaning
List<XmagicProperty<?>> assetsList
List of animated effect materials to be checked.

getPropertyRequiredAbilities

Inputs an animated effect resources list, returns the SDK atomic abilities list used for each resource. The usage scenario of this method: If you have purchased or created a number of animated effect materials, by invoking this method, it will returning a list of atomic abilities required for each material. For instance, material 1 requires abilities A, B, C while material 2 need abilities B, C, D. Subsequently, you maintain such a list of abilities on your server. Later on, when a user wants to download the animated effect materials from the server, the user first accesses the list of atomic abilities that his mobile has through the getDeviceAbilities method (for example, this phone possess abilities A, B, C, but lacks D), transmits this abilities list to the server. The server, on determining the device does not have ability D, therefore does not issue material 2 to the user.

Parameter

Parameter
Meaning
List<XmagicProperty<?>> assets
List of animated effect resources for which to check the atomic capabilities.

Return

Returned value Map<XmagicProperty<?>,ArrayList<String>> :
key: animated effect resource material entity class.
value: list of used atomic capabilities.

getDeviceAbilities

It returns the atomic capability table that the current device supports. To be used in conjunction with the getPropertyRequiredAbilities method. For more details, please refer to the description of the getPropertyRequiredAbilities.
Map<String,Boolean> getDeviceAbilities()

Return

Return value Map<String,Boolean>:
key: atomic capability name (corresponding to the material capability name).
value: whether it is supported by the current device.

isSupportBeauty

Determine whether the current device supports refinement (OpenGL3.0).
boolean isSupportBeauty()

Return

Return value boolean: Determines whether the beauty filter is supported.

isBeautyAuthorized

Determine which beauty or body modifier features the current License authorization supports. Only detection of BEAUTY and BODY_BEAUTY types are supported. The resultant evaluation will be allocated to each beauty object's XmagicProperty.isAuth field. If the isAuth field returns false, the entrances for these features can be concealed in the UI.
void isBeautyAuthorized(List<XmagicProperty<?>> properties)

Parameter

Parameter
Meaning
List<XmagicProperty<?>> properties
Beauty filters that need to be checked.

setXmagicStreamType

Sets the input data type, which is Android camera data stream (XmagicApi.PROCESS_TYPE_CAMERA_STREAM) by default.
void setXmagicStreamType(int type)

Parameter

Parameter
Meaning
int type
Data source type, there are two options:
XmagicApi.PROCESS_TYPE_CAMERA_STREAM: camera data source.
XmagicApi.PROCESS_TYPE_PICTURE_DATA: Image data source.

setXmagicLogLevel

Setting the log level of the SDK, the default is WARN. During the development debugging stage if necessary, it can be set to Log.DEBUG.. When officially released, it must be set to Log.WARN or Log.ERROR otherwise, an immense amount of logs will adversely affect performance.
Invoke after new XmagicApi().

setAudioMute

Whether to mute when using animation materials (Added in V2.5.0):
Parameter: true indicates mute, false denotes non-mute.

enableEnhancedMode

Activate the beautification enhancing pattern (Added in V2.5.1). By default, it is not activated.
When inactive, the strength range of each beauty option in the application layer can be set from 0 to 1 or -1 to 1. If this range is exceeded, the SDK will take the boundary value. For instance, if the application layer sets the face-thinning to 1.2, the SDK determines that this surpasses the maximum value of 1.0, and internally adjusts the face-thinning value to 1.0.
After enabling the enhanced pattern, the application layer can set a broader range of values. For instance, if a greater face slimming effect is desired, you can set the slimming value to 1.2. The SDK will accept and use this value of 1.2 without correcting it to 1.0.
Upon activation of the enhanced pattern, it becomes the responsibility of the application layer to manage the maximal values of each beauty setting, allowing users to adjust within this range. We provide a set of reference values which you may adjust freely according to product requirements, though we advise not exceeding our recommended values to avoid potential degradation of beauty effects. The reference values are as follows:
Beauty Item Name
In enhanced pattern, recommended maximum value (magnification factor)
Whitening, shortening the face, V-face, eye distance, nose position, removal of laugh lines, lipstick, three-dimensional appearance
1.3
Eye lightening
1.5
Blush
1.8
Other
1.2

setDowngradePerformance(Added in V3.1.0)

Upon activating the advanced performance pattern, the resources consumed by the beautification function on your system's CPU/GPU are diminished, effectively mitigating the heating and lagging issues of your mobile device, making it particularly beneficial for extended utilization on lower-end devices.
Note: After enabling the high performance pattern, the following beauty options will be unavailable:
1. Ocular region: Eye width, eye height, removal of eye bags .
2. Eyebrows: Angle, Distance, Height, Length, Thickness, Eyebrow Peak.
3. Mouth: The Smiling Lips.
4. Facial: Slimming (natural, goddess, handsome), jaw reduction, wrinkle removal, and nasolabial fold removal. It is recommended to use "face shape" to achieve the comprehensive effect of large eyes and slim face.
void boolean setDowngradePerformance()

exportCurrentTexture

Get the picture on the current texture
void exportCurrentTexture(ExportTextureCallback callback)

setFeatureEnableDisable

Enable or disable a feature.
void setFeatureEnableDisable(String featureName, boolean enable)

Parameter

Parameter
Meaning
String featureName
feature Name
Values:
XmagicConstant.FeatureName.ANIMOJI_52_EXPRESSION facial expressions feature.
XmagicConstant.FeatureName.BODY_3D_POINT 3D body data feature.
XmagicConstant.FeatureName.HAND_DETECT gesture detection.
XmagicConstant.FeatureName.WHITEN_ONLY_SKIN_AREA Brightening only applies to skin.
XmagicConstant.FeatureName.SEGMENTATION_SKIN segmentation skin.
XmagicConstant.FeatureName.SMART_BEAUTY smart beauty(reducing the intensity of beauty and makeup effects for males and babies).
boolean enable
"true" indicates enabling a capability, while "false" indicates disabling a capability.
Note: If it is in downgrade mode, enabling skin segmentation is not allowed.

Static properties and methods

API
Description
VERSION
The SDK version number can be obtained through XmagicApi.VERSION (Added in V3.5.0)
Setting libPath.
Transfer the content located within the directories Light3DPlugin, LightCore, LightHandPlugin, LightBodyPlugin, LightSegmentPlugin under application assets to your designated directory.
Copy the AI model files downloaded by the client to the corresponding folders.

setLibPathAndLoad

Setting the path for SO and triggering a load. If SO is built into the assets, there is no need to invoke this method. If SO is dynamically downloaded, it must be invoked before authentication and new XmagicApi.
Passing in null indicates loading the so from the default path. Please ensure that the so is built into the APK package.
Pass in non-null: such as data/data/package name/files/xmagic_libs, the so will be loaded from this directory.
static boolean setLibPathAndLoad(String path)

Parameter

Parameter
Meaning
String path
so: denotes the path to the library.

addAiModeFilesFromAssets

Transfer the content located within the directories Light3DPlugin, LightCore, LightHandPlugin, LightBodyPlugin, LightSegmentPlugin under application assets to your designated directory.
context applicationcontext.
resDir serves as the root directory for storing beautification resources. This directory corresponds to the path passed when creating an xmagicApi object.
Returned values:
0: Copy successful
-1: denotes that the context is null
-2: denotes an IO error
static int addAiModeFilesFromAssets(Context context, String resDir)

Parameter

Parameter
Meaning
Context context
applicationcontext.
String resDir
Designated as the root directory for storing beautification resources, this directory corresponds to the path input when creating the xmagicApi object.

addAiModeFiles

Copy the AI model files downloaded by the client to the corresponding folders.
inputResDir is the directory of the successfully downloaded model files.
resDir serves as the root directory for storing beautification resources. This directory corresponds to the path passed when creating an xmagicApi object.
Returned values:
0: denotes success
-1:inputResDir is not exists
-2: Signifies an IO error
static int addAiModeFiles(String inputResDir, String resDir)

Parameter

Parameter
Meaning
String inputResDir
The folder containing the successfully downloaded model files.
String resDir
Designated as the root directory for storing beautification resources, this directory corresponds to the path input when creating the xmagicApi object.