当前位置:网站首页>Threedposetracker project resolution
Threedposetracker project resolution
2022-07-06 01:12:00 【Boiled Green Vegetable】
Catalog
(2) Theoretical analysis of algorithm
3、 ... and 、 Limb drive flow chart
One 、 Source code
Digital Standard Co., LTD. (github.com)
Reference article :
- Cartoon driven project ThreeDPoseTracker—— Key point smoothing scheme analysis
- 3D Attitude estimation ——ThreeDPose Easy to use model analysis of the project
plug-in unit :ThreeDPoseTracker Source related plug-ins
Two 、 Body drive
1. Algorithm model
(1) Data description
The code of algorithm model prediction data in the project is mainly VNectBarracudaRunner.cs. function PredictPose() The updated node data in is JointPoint Three variables in ( That is, the algorithm data to be used when driving ):
public class JointPoint
{
public Vector3 Pos3D = new Vector3();
public float score3D;
public bool Visibled;
}
JointPoint There are also several variables in the class that are used for data optimization in the prediction process ( Anti jitter, etc ) Of :
public class JointPoint
{
public Vector3 Now3D = new Vector3();
public Vector3[] PrevPos3D = new Vector3[6];
public Vector3 P = new Vector3();
public Vector3 X = new Vector3();
public Vector3 K = new Vector3();
}
JointPoint Other data in the class need to be calculated by yourself when driven by limbs :
public class JointPoint
{
public Transform Transform = null;
public Quaternion InitRotation;
public Quaternion Inverse;
public Quaternion InverseRotation;
public JointPoint Child = null;
public JointPoint Parent = null;
}
(2) Theoretical analysis of algorithm
Resolve jitter : Kalman filtering + Low pass filtering : Key smoothing scheme
Algorithm model analysis : Analytical model 、 heatmap&offset
- There are three input, It's all the same , All input (448,448,3) Pictures of the :
private void UpdateVNectAsync()
{
input = new Tensor(videoCapture.MainTexture, 3);
if (inputs[inputName_1] == null)
{
inputs[inputName_1] = input;
inputs[inputName_2] = new Tensor(videoCapture.MainTexture, 3);
inputs[inputName_3] = new Tensor(videoCapture.MainTexture, 3);
}
else
{
inputs[inputName_3].Dispose();
inputs[inputName_3] = inputs[inputName_2];
inputs[inputName_2] = inputs[inputName_1];
inputs[inputName_1] = input;
}
if (!Lock && videoCapture.IsPlay())
{
StartCoroutine(ExecuteModelAsync());
}
}
- four output, But we only use the last two :
for (var i = 2; i < _model.outputs.Count; i++)
{
b_outputs[i] = _worker.PeekOutput(_model.outputs[i]);
}
offset3D = b_outputs[2].data.Download(b_outputs[2].shape);
heatMap3D = b_outputs[3].data.Download(b_outputs[3].shape);
use heatmap Roughly position the joint , And then use offset stay heatmap As a result, the joint position is adjusted accurately .
heatmap Of ( 672 , 28 , 28 ) representative 24 With two joints 28 Size is (28,28) Characteristic graph . and offset Than heatmap Three times more characteristic graphs , Obviously, it is the precise positioning just mentioned , It just needs to be offset To x,y,z Three coordinates , So it's a triple relationship .
- heatmap The order of is 1 The first joint 1 A feature map 、 The first 1 The first joint 2 A feature map 、…、 The first 2 The first joint 1 A feature map 、 The second joint 2 A feature map 、…、 The first 24 The first joint 28 A feature map
- offsetmap The order of is 1 The first joint 1 Corresponding to each characteristic graph x Coordinate offset 、 The first 1 The first joint 2 Corresponding to each characteristic graph x Coordinate offset 、 The first 1 The first joint 3 Corresponding to each characteristic graph x Coordinate offset 、…、 The first 1 The first joint 28 Corresponding to each characteristic graph x Coordinate offset 、…、 The first 2 The first joint 1 Corresponding to each characteristic graph x Coordinate offset 、…、 The first 24 The first joint 28 Corresponding to each characteristic graph x Coordinate offset 、 The first 1 The first joint 1 Corresponding to each characteristic graph y Coordinate offset 、 The first 1 The first joint 2 Corresponding to each characteristic graph y Coordinate offset 、…、 The first 24 The first joint 28 Corresponding to each characteristic graph y Coordinate offset 、 The first 1 The first joint 1 Corresponding to each characteristic graph z Coordinate offset 、 The first 1 The first joint 2 Corresponding to each characteristic graph z Coordinate offset 、…、 The first 24 The first joint 28 Corresponding to each characteristic graph z Coordinate offset .
2.Unity Body drive
The body driven code in the project is mainly VNectModel.cs.
(1) Related knowledge
- The inverse of four elements ,⽐ Such as ⼀ Quaternions are (1,1,1,1) Its inverse is (-1,-1,-1,1).
Quaternion Inverse(Quaternion rotation);
return ⼀ An opposite rotation .
- Watch the rotation
public static Quaternion LookRotation(Vector3 forward, [DefaultValue("Vector3.up")] Vector3 upwards);
Create a specific forward and upward Direction of rotation , Returns a computed quaternion . If used to determine a transform: If the vector is orthogonal , z The axis will be and forward alignment ,y The axis will be and upward alignment . If forward The direction is 0 Will be an error .
(2) Use in projects
from Of the current joint lookrotation= Initial rotation InitRotation× Align matrix And It can be concluded that :
a. Quaternion.Inverse( Align matrix )= Current rotation Rotation*Quaternion.Inverse( Of the current joint lookrotation)
hip.Inverse = Quaternion.Inverse(Quaternion.LookRotation(forward));
hip.InverseRotation = hip.Inverse * hip.InitRotation;
function Init() There are a lot of similar code in , The purpose is Find the intermediate matrix , According to the intermediate matrix , We can get the current object The rotation of the .VNectModel.cs Medium hip.InverseRotation Namely Quaternion.Inverse( Align matrix ).
b. Current rotation Rotation= Of the current joint lookrotation×Quaternion.Inverse( Align matrix )
// Calculate the orientation of the human body based on the coordinates of the root joint and the left and right crotch joints , Then use this as all joints LookRotation Of y Direction
var forward = TriangleNormal(jointPoints[PositionIndex.hip.Int()].Pos3D, jointPoints[PositionIndex.lThighBend.Int()].Pos3D, jointPoints[PositionIndex.rThighBend.Int()].Pos3D);
jointPoints[***].Transform.rotation = Quaternion.LookRotation(Vector3.up, forward) * jointPoint[***].InverseRotation;
function PoseUpdate() For limb drive , The above line of code is used to calculate the current rotation of the joint .
c. Align matrix = Of the current joint lookrotation*Quaternion.Inverse( Current rotation Rotation);
root = animator.GetBoneTransform(HumanBodyBones.Hips);
midRoot = Quaternion.Inverse(root.rotation) * Quaternion.LookRotation(forward);//midRoot To align the matrix
These codes do not exist in the project , Just to better explain the formula .
3、 ... and 、 Limb drive flow chart
边栏推荐
- 95后CV工程师晒出工资单,狠补了这个,真香...
- Vulhub vulnerability recurrence 74_ Wordpress
- ORA-00030
- 激动人心,2022开放原子全球开源峰会报名火热开启
- Live broadcast system code, custom soft keyboard style: three kinds of switching: letters, numbers and punctuation
- Study diary: February 13, 2022
- Distributed base theory
- VSphere implements virtual machine migration
- I'm interested in watching Tiktok live beyond concert
- A preliminary study of geojson
猜你喜欢
Study diary: February 13, 2022
Keepalive component cache does not take effect
Mobilenet series (5): use pytorch to build mobilenetv3 and learn and train based on migration
JVM_ 15_ Concepts related to garbage collection
Some features of ECMAScript
Cf:d. insert a progression [about the insert in the array + the nature of absolute value + greedy top-down]
Recoverable fuse characteristic test
Daily practice - February 13, 2022
Test de vulnérabilité de téléchargement de fichiers basé sur dvwa
MCU通过UART实现OTA在线升级流程
随机推荐
IP storage and query in MySQL
File upload vulnerability test based on DVWA
Development trend of Ali Taobao fine sorting model
Xunrui CMS plug-in automatically collects fake original free plug-ins
Nmap: network detection tool and security / port scanner
Kotlin basics 1
Beginner redis
毕设-基于SSM高校学生社团管理系统
Four commonly used techniques for anti aliasing
Questions about database: (5) query the barcode, location and reader number of each book in the inventory table
curlpost-php
DOM introduction
golang mqtt/stomp/nats/amqp
The population logic of the request to read product data on the sap Spartacus home page
Hundreds of lines of code to implement a JSON parser
[groovy] JSON string deserialization (use jsonslurper to deserialize JSON strings | construct related classes according to the map set)
记一个 @nestjs/typeorm^8.1.4 版本不能获取.env选项问题
[day 30] given an integer n, find the sum of its factors
Opinions on softmax function
Daily practice - February 13, 2022