当前位置:网站首页>Threedposetracker project resolution
Threedposetracker project resolution
2022-07-06 01:12:00 【Boiled Green Vegetable】
Catalog
(2) Theoretical analysis of algorithm
3、 ... and 、 Limb drive flow chart
One 、 Source code
Digital Standard Co., LTD. (github.com)
Reference article :
- Cartoon driven project ThreeDPoseTracker—— Key point smoothing scheme analysis
- 3D Attitude estimation ——ThreeDPose Easy to use model analysis of the project
plug-in unit :ThreeDPoseTracker Source related plug-ins
Two 、 Body drive
1. Algorithm model
(1) Data description
The code of algorithm model prediction data in the project is mainly VNectBarracudaRunner.cs. function PredictPose() The updated node data in is JointPoint Three variables in ( That is, the algorithm data to be used when driving ):
public class JointPoint
{
public Vector3 Pos3D = new Vector3();
public float score3D;
public bool Visibled;
}
JointPoint There are also several variables in the class that are used for data optimization in the prediction process ( Anti jitter, etc ) Of :
public class JointPoint
{
public Vector3 Now3D = new Vector3();
public Vector3[] PrevPos3D = new Vector3[6];
public Vector3 P = new Vector3();
public Vector3 X = new Vector3();
public Vector3 K = new Vector3();
}
JointPoint Other data in the class need to be calculated by yourself when driven by limbs :
public class JointPoint
{
public Transform Transform = null;
public Quaternion InitRotation;
public Quaternion Inverse;
public Quaternion InverseRotation;
public JointPoint Child = null;
public JointPoint Parent = null;
}
(2) Theoretical analysis of algorithm
Resolve jitter : Kalman filtering + Low pass filtering : Key smoothing scheme
Algorithm model analysis : Analytical model 、 heatmap&offset
- There are three input, It's all the same , All input (448,448,3) Pictures of the :
private void UpdateVNectAsync()
{
input = new Tensor(videoCapture.MainTexture, 3);
if (inputs[inputName_1] == null)
{
inputs[inputName_1] = input;
inputs[inputName_2] = new Tensor(videoCapture.MainTexture, 3);
inputs[inputName_3] = new Tensor(videoCapture.MainTexture, 3);
}
else
{
inputs[inputName_3].Dispose();
inputs[inputName_3] = inputs[inputName_2];
inputs[inputName_2] = inputs[inputName_1];
inputs[inputName_1] = input;
}
if (!Lock && videoCapture.IsPlay())
{
StartCoroutine(ExecuteModelAsync());
}
}
- four output, But we only use the last two :
for (var i = 2; i < _model.outputs.Count; i++)
{
b_outputs[i] = _worker.PeekOutput(_model.outputs[i]);
}
offset3D = b_outputs[2].data.Download(b_outputs[2].shape);
heatMap3D = b_outputs[3].data.Download(b_outputs[3].shape);
use heatmap Roughly position the joint , And then use offset stay heatmap As a result, the joint position is adjusted accurately .
heatmap Of ( 672 , 28 , 28 ) representative 24 With two joints 28 Size is (28,28) Characteristic graph . and offset Than heatmap Three times more characteristic graphs , Obviously, it is the precise positioning just mentioned , It just needs to be offset To x,y,z Three coordinates , So it's a triple relationship .
- heatmap The order of is 1 The first joint 1 A feature map 、 The first 1 The first joint 2 A feature map 、…、 The first 2 The first joint 1 A feature map 、 The second joint 2 A feature map 、…、 The first 24 The first joint 28 A feature map
- offsetmap The order of is 1 The first joint 1 Corresponding to each characteristic graph x Coordinate offset 、 The first 1 The first joint 2 Corresponding to each characteristic graph x Coordinate offset 、 The first 1 The first joint 3 Corresponding to each characteristic graph x Coordinate offset 、…、 The first 1 The first joint 28 Corresponding to each characteristic graph x Coordinate offset 、…、 The first 2 The first joint 1 Corresponding to each characteristic graph x Coordinate offset 、…、 The first 24 The first joint 28 Corresponding to each characteristic graph x Coordinate offset 、 The first 1 The first joint 1 Corresponding to each characteristic graph y Coordinate offset 、 The first 1 The first joint 2 Corresponding to each characteristic graph y Coordinate offset 、…、 The first 24 The first joint 28 Corresponding to each characteristic graph y Coordinate offset 、 The first 1 The first joint 1 Corresponding to each characteristic graph z Coordinate offset 、 The first 1 The first joint 2 Corresponding to each characteristic graph z Coordinate offset 、…、 The first 24 The first joint 28 Corresponding to each characteristic graph z Coordinate offset .
2.Unity Body drive
The body driven code in the project is mainly VNectModel.cs.
(1) Related knowledge
- The inverse of four elements ,⽐ Such as ⼀ Quaternions are (1,1,1,1) Its inverse is (-1,-1,-1,1).
Quaternion Inverse(Quaternion rotation);
return ⼀ An opposite rotation .
- Watch the rotation
public static Quaternion LookRotation(Vector3 forward, [DefaultValue("Vector3.up")] Vector3 upwards);
Create a specific forward and upward Direction of rotation , Returns a computed quaternion . If used to determine a transform: If the vector is orthogonal , z The axis will be and forward alignment ,y The axis will be and upward alignment . If forward The direction is 0 Will be an error .
(2) Use in projects
from Of the current joint lookrotation= Initial rotation InitRotation× Align matrix And It can be concluded that :
a. Quaternion.Inverse( Align matrix )= Current rotation Rotation*Quaternion.Inverse( Of the current joint lookrotation)
hip.Inverse = Quaternion.Inverse(Quaternion.LookRotation(forward));
hip.InverseRotation = hip.Inverse * hip.InitRotation;
function Init() There are a lot of similar code in , The purpose is Find the intermediate matrix , According to the intermediate matrix , We can get the current object The rotation of the .VNectModel.cs Medium hip.InverseRotation Namely Quaternion.Inverse( Align matrix ).
b. Current rotation Rotation= Of the current joint lookrotation×Quaternion.Inverse( Align matrix )
// Calculate the orientation of the human body based on the coordinates of the root joint and the left and right crotch joints , Then use this as all joints LookRotation Of y Direction
var forward = TriangleNormal(jointPoints[PositionIndex.hip.Int()].Pos3D, jointPoints[PositionIndex.lThighBend.Int()].Pos3D, jointPoints[PositionIndex.rThighBend.Int()].Pos3D);
jointPoints[***].Transform.rotation = Quaternion.LookRotation(Vector3.up, forward) * jointPoint[***].InverseRotation;
function PoseUpdate() For limb drive , The above line of code is used to calculate the current rotation of the joint .
c. Align matrix = Of the current joint lookrotation*Quaternion.Inverse( Current rotation Rotation);
root = animator.GetBoneTransform(HumanBodyBones.Hips);
midRoot = Quaternion.Inverse(root.rotation) * Quaternion.LookRotation(forward);//midRoot To align the matrix
These codes do not exist in the project , Just to better explain the formula .
3、 ... and 、 Limb drive flow chart
边栏推荐
- View class diagram in idea
- 基于DVWA的文件上传漏洞测试
- Pbootcms plug-in automatically collects fake original free plug-ins
- Nmap: network detection tool and security / port scanner
- JVM_ 15_ Concepts related to garbage collection
- Live broadcast system code, custom soft keyboard style: three kinds of switching: letters, numbers and punctuation
- curlpost-php
- [groovy] compile time metaprogramming (compile time method injection | method injection using buildfromspec, buildfromstring, buildfromcode)
- The inconsistency between the versions of dynamic library and static library will lead to bugs
- Intensive learning weekly, issue 52: depth cuprl, distspectrl & double deep q-network
猜你喜欢
基于DVWA的文件上传漏洞测试
Hcip---ipv6 experiment
Illustrated network: the principle behind TCP three-time handshake, why can't two-time handshake?
Condition and AQS principle
Study diary: February 13, 2022
Installation and use of esxi
毕设-基于SSM高校学生社团管理系统
Five challenges of ads-npu chip architecture design
Daily practice - February 13, 2022
BiShe - College Student Association Management System Based on SSM
随机推荐
VMware Tools installation error: unable to automatically install vsock driver
The basic usage of JMeter BeanShell. The following syntax can only be used in BeanShell
cf:H. Maximal AND【位运算练习 + k次操作 + 最大And】
Gartner发布2022-2023年八大网络安全趋势预测,零信任是起点,法规覆盖更广
ORA-00030
Novice entry depth learning | 3-6: optimizer optimizers
Zhuhai's waste gas treatment scheme was exposed
[groovy] XML serialization (use markupbuilder to generate XML data | create sub tags under tag closures | use markupbuilderhelper to add XML comments)
Construction plan of Zhuhai food physical and chemical testing laboratory
视频直播源码,实现本地存储搜索历史记录
测试/开发程序员的成长路线,全局思考问题的问题......
Test de vulnérabilité de téléchargement de fichiers basé sur dvwa
Intensive learning weekly, issue 52: depth cuprl, distspectrl & double deep q-network
cf:C. The Third Problem【关于排列这件事】
从 1.5 开始搭建一个微服务框架——调用链追踪 traceId
Introduction to robotics I. spatial transformation (1) posture, transformation
[day 30] given an integer n, find the sum of its factors
Some features of ECMAScript
Kotlin core programming - algebraic data types and pattern matching (3)
Study diary: February 13, 2022