This capability processes the OpenGL textures of the camera and outputs 3D body data. You can pass the data to Unity to drive your model or use the data to implement other features.
voidonAIDataUpdated(String data);//This method is a new method added in version 3.0.0, and the data structure is consistent with the XmagicYTDataListener interface used in previous versions.
}
onAIDataUpdated returns a JSON string. You can find an example below.
face_info is facial data, which is not used by this capability.
For the meanings of fields in body_3d_info, see below.
Body Keypoints and Descriptions
SMPL keypoints.
SMPL-X hand keypoints
Here is an example of the JSON string output by the SDK:
The following section details the fields in body_3d_info:
imageWidth and imageHeight: The width and height of images sent to the SDK.
items: An array. Currently, it contains only one element.
index: A reserved field. You can ignore it.
pose:
Location [0,2]. The 3D location (xyz) of the body root bone with the camera at the center.
Location [3,12]. The body shape. It includes 10 floating numbers, which are based on 10 meshes of SMPL.
Location [13]. The focal length, which is 5000.
Location [14,29]. The OpenGL projection matrix in a 3D space, which is generated based on the focal length. A 4 x 4 projection matrix is calculated as follows in the algorithm:
matrix={
2 * focal_length / img_wid, 0, 0, 0,
0, 2 * focal_length / img_hei, 0,0,
0,0, (zf + zn) / (zn - zf), -1,
0, 0, (2.0f * zf * zn) / (zn - zf), 0};
}
Location [30,33]. Whether the left toe, left heel, right toe, and right heel are on the ground.
position_x,position_y,position_z：
Location [0,23]. 2D body keypoints (figure 1 above). The value of position_z for all 2D keypoints is 0.
Location [24,47]. 3D body keypoints (figure 1 above).
rotation
Location [0,23]. The body bone rotation quaternions (wxyz).
Location [25,54]. The hand bone rotation quaternions (wxyz). There are 15 quaternions for each hand.
