Unity | two ways to realize facial drive

Catalog

One 、 Application unity Live Capture Plug in and iphone Unity Face Capture Software

1. Version for

2. unity In the operation ：

（1） introduce Live Capture Package

（2） Create an empty object （ name ：Recorder）, add to Take Recorder Components .

 （3） Drag the model into the scene , And add ARKit Face Actor Components , Make the model into Prefab.

（4） Set up the model ARKit Face Actor In the component mapper.

（5） Set up New FaceDevice Of ARKit Face Device Component's Actor value .

（6） Create a server .

3. iphone download Unity Face Capture Software

Two 、 Control through algorithm data BlendShape The number

1. Correlation function

2. Realization

---

One 、 Application unity Live Capture Plug in and iphone Unity Face Capture Software

1. Version for

• unity edition ：Unity Editor 2020.3.16f1 or later version
• iphone edition ：iOS 14.6 or higher
• Reference documents ：About Live Capture | Live Capture | 1.0.1 (unity3d.com)

2. unity In the operation ：

（1） introduce Live Capture Package

        Live Capture stay Package Manager Pass through git url How to add ：

（2） Create an empty object （ name ：Recorder）, add to Take Recorder Components .

        Add ARKit Face Device： It will be in Recorder Automatically create under the object New FaceDevice, There is a ARKit Face Device Components .

 （3） Drag the model into the scene , And add ARKit Face Actor Components , Make the model into Prefab.

        Pay attention to model import Unity Check... When Import BlendShapes（ Just check it ,unity Check... By default ）.

        stay Project Create a mapper( Right click /Create/Live Capture / ARKit Face Capture / Mapper), Set up mapper Related properties （RigPrefab Specify as generated in the previous step prefab,LeftEye、RightEye、Head To select the corresponding joint . The purpose of this step is to map BlendShape, If our model BlendShape The name matches Apple's ARKit requirement , be AddRender The operation will be mapped automatically , Otherwise, we need to set it manually ）.

（4） Set up the model ARKit Face Actor In the component mapper.

（5） Set up New FaceDevice Of ARKit Face Device Component's Actor value .

（6） Create a server .

        Click on Window/Live Capture/Connections, Click on Configure FireWall. Click on Create Server/Companion App Server, Click on Start Opening service .

3. iphone download Unity Face Capture Software

         Open software （ The software is in iphone 12 Pro Max 15.4.1 Upper pro test stability , stay iphone X It will heat up and flash back ）, Input IP And Port, After connecting, you can drive ~（ If you can't connect , Probably unity The version does not conform or the network does not conform , Refer to official documents ：About Live Capture | Live Capture | 1.0.1 (unity3d.com)）

Two 、 Control through algorithm data BlendShape The number .

1. Correlation function

• obtain BlendShape Quantity and corresponding name .

public sealed class Mesh : Object
{
    public int blendShapeCount { get; }
    public string GetBlendShapeName(int shapeIndex);
}

• Set up BlendShape The weight

public class SkinnedMeshRenderer : Renderer{
    public void SetBlendShapeWeight(int index, float value);
}

2. Realization

         The data value range provided by the algorithm may be 0-1, So in SetBlendShapeWeight When it comes to *100.

Dictionary<string, int> bs_map;// On the model bs name - Indexes 
skinnedMeshRenderer.SetBlendShapeWeight(bs_map["eyebrow_BS.Eyebrows_Sad_L"], 20);