iOS

功能说明


输入音频数据,输出苹果 ARKit 标准的52表情数据,请参见 ARFaceAnchor。 您可以利用这些表情数据做一进步的开发,例如传到 Unity 中驱动您的模型。

接入方式

方式1:与腾讯特效 SDK 一起使用

1. 语音转表情sdk结合腾讯特效SDK中,因此第一步需要按照腾讯特效文档进行接入。
3. 参考 独立集成腾讯特效 文档完成集成。
4. 腾讯特效 SDK 完整版内的Audio2Exp.framework 拉入项目,并且在项目的target->General->Frameworks,Libraries,and Embedded Content处设置为Embed & Sign。

方式2:通过独立的语音转表情 SDK 接入

如果您只需要语音转表情,不需要用到腾讯特效 SDK 的任何能力,则可以考虑使用独立的语音转表情 SDK,Audio2Exp.framework 包约7MB左右。项目引入Audio2Exp.framework、YTCommonXMagic.framework两个动态库,并且在项目的target->General->Frameworks,Libraries,and Embedded Content处设置为Embed & Sign

接入步骤

1. 设置 License,请参见 鉴权
2. 配置模型文件:请将必需的模型文件audio2exp.bundle拷贝到工程目录,然后在调用 Audio2ExpApi 的 initWithModelPath: 接口时,传入参数 audio2exp.bundle" 模型文件所在的路径。

接口说明

接口
说明
+ (int)initWithModelPath:(NSString*)modelPath;
初始化,传入模型路径,见上文说明。返回值为0表示成功
+ (NSArray *)parseAudio:(NSArray *)inputData;
输入的是音频数据,要求单通道,16K采样率,数组长度为267(即267个采样点),输出的数据是长度为52的float数组,表示52表情基,取值为0到1之间,顺序为 苹果标准顺序{"eyeBlinkLeft","eyeLookDownLeft","eyeLookInLeft","eyeLookOutLeft","eyeLookUpLeft","eyeSquintLeft","eyeWideLeft","eyeBlinkRight","eyeLookDownRight","eyeLookInRight","eyeLookOutRight","eyeLookUpRight","eyeSquintRight","eyeWideRight","jawForward","jawLeft","jawRight","jawOpen","mouthClose","mouthFunnel","mouthPucker","mouthRight","mouthLeft","mouthSmileLeft","mouthSmileRight","mouthFrownRight","mouthFrownLeft","mouthDimpleLeft","mouthDimpleRight","mouthStretchLeft","mouthStretchRight","mouthRollLower","mouthRollUpper","mouthShrugLower","mouthShrugUpper","mouthPressLeft","mouthPressRight","mouthLowerDownLeft","mouthLowerDownRight","mouthUpperUpLeft","mouthUpperUpRight","browDownLeft","browDownRight","browInnerUp","browOuterUpLeft","browOuterUpRight","cheekPuff","cheekSquintLeft","cheekSquintRight","noseSneerLeft","noseSneerRight","tongueOut"}
+ (int)releaseSdk
使用完毕后调用,释放资源

集成代码示例

// 初始化语音转表情sdk
NSString *path = [[NSBundle mainBundle] pathForResource:@"audio2exp" ofType:@"bundle"];
int ret = [Audio2ExpApi initWithModelPath:path];
// 语音数据转52表情数据
NSArray *emotionArray = [Audio2ExpApi parseAudio:floatArr];
// 释放sdk
[Audio2ExpApi releaseSdk];

// 结合腾讯特效sdk xmgaic使用
// 使用对应的资源初始化美颜sdk
self.beautyKit = [[XMagic alloc] initWithRenderSize:previewSize assetsDict:assetsDict];
// 加载avatar素材
[self.beautyKit loadAvatar:bundlePath exportedAvatar:nil completion:nil];
// 将52表情数据传入美颜sdk 就能看到效果
[self.beautyKit updateAvatarByExpression:emotionArray];
说明:
完整的示例代码请参考 美颜特效 SDK demo 工程
录音可以参考 TXCAudioRecorder
接口使用可以参考:VoiceViewController 及其相关类。