当前位置:网站首页>[happy Spring Festival] if you feel happy, dance
[happy Spring Festival] if you feel happy, dance
2022-07-06 05:46:00 【Python's path to immortality】
Project introduction
This project builds a simple and practical dance machine based on the pose estimation model .
Project use PyQT5 Designed a simple interactive interface , The original dance video is displayed on the left , The camera is displayed on the right , Here I use a built-in camera in my notebook , Friends can change other cameras or projectors , May experience better .
Be careful : All materials are from the Internet , If there is infringement , Please contact to delete
B Station video experience is as follows :
b Station video link :https://www.bilibili.com/video/BV1am4y1f7c2/
Interface design
The interface layout is mainly divided into 4 part , The upper middle shows the number of correct posture matching :
- On the top left is the drum beat, which always refers to the posture of the dance
- Below the left is the reference dance video source
- On the top right is the posture of the real-time video when the drum is beating
- On the lower right is the real-time video source of the camera
Menu management
Right click in the reference dance video source area to display the menu , It mainly includes
- Camera settings
- Song management
- Start the game ( Before the game begins )
- Stop swimming at the beginning ( When the game is running )
- Full screen display
- sign out
Click song management to add or delete dance videos , You can also switch the currently selected song in the song list .
Preload dance videos
After adding a dance video , The background of the program will automatically analyze the drum time of the dance video and the corresponding posture description .
Use librosa Ku analyzes the drums of dance songs , Finally, it is converted into the frame number and sequence number corresponding to the video .
y, sr = librosa.load(video_path)
_, beats = librosa.beat.beat_track(y=y, sr=sr)
beat_times = list(librosa.frames_to_time(beats, sr=sr))
beat_times.append(beat_times[-1] + 1)
frameIndex = [round(bt * 25) for bt in beat_times]
After preloading the dance video, it will generate PKL file , Stored in projects/songs in , The dictionary format is as follows :
data = {
"path": video_path,
"fps": capture.get(cv2.CAP_PROP_FPS),
"width": int(capture.get(cv2.CAP_PROP_FRAME_WIDTH)),
"height": int(capture.get(cv2.CAP_PROP_FRAME_HEIGHT)),
"beats": frameIndex,
"poses": [],
"pose_imgs": [],
"start_ends": start_ends,
"count": 0
}
Attitude estimation
Due to the high real-time requirements of human-computer interaction for model reasoning , After investigating many models , The final model selection is PaddleDetection Open source PicoDet-S-Pedestrian as well as PP-TinyPose, Model reasoning time single frame 20ms about , The speed and effect can meet the requirements .
PP-TinyPose yes PaddleDetecion Real time attitude detection model optimized for mobile devices , It can smoothly perform multi person pose estimation tasks on mobile devices . With the help of PaddleDetecion Self developed excellent lightweight detection model PicoDet, We also provide a characteristic lightweight pedestrian detection model .
PP-TinyPose link : https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/keypoint/tiny_pose
Attitude matching algorithm
After the estimation of human posture , Compare the pose of the real-time camera with the reference dance video . In practice, it is difficult to synchronize the action time of the two , Here is a simple treatment , It is acceptable not to exceed 10 In the frame range .
The actual comparison is the relevant bone connecting lines on the left and right sides of the human body , In particular, the included angle of the connecting line is compared , stay 15 Within the range of degrees, it is considered to be similar .
The current matching algorithm is very rough , Many further optimizations are needed , Especially in similarity judgment , Key points are adopted in the later stage Graph The way may work better .
The calculation logic of angle matching is as follows :
def angle_match(self, pose_ref, pose_cur, points_index=()):
if len(points_index) != 3:
return False
for i in range(3):
idx = points_index[i]
if pose_ref[idx][2] < 0.3:
print("pose ref true")
return True
def get_angle(pose):
point_1 = pose[points_index[0]]
point_2 = pose[points_index[1]]
point_3 = pose[points_index[2]]
a = math.sqrt((point_2[0] - point_3[0]) * (point_2[0] - point_3[0]) + (point_2[1] - point_3[1]) * (
point_2[1] - point_3[1]))
b = math.sqrt((point_1[0] - point_3[0]) * (point_1[0] - point_3[0]) + (point_1[1] - point_3[1]) * (
point_1[1] - point_3[1]))
c = math.sqrt((point_1[0] - point_2[0]) * (point_1[0] - point_2[0]) + (point_1[1] - point_2[1]) * (
point_1[1] - point_2[1]))
return math.degrees(math.acos((b * b - a * a - c * c) / (-2 * a * c)))
return abs(get_angle(pose_ref) - get_angle(pose_cur)) < 15
def pose_match(self, pose_ref, pose_cur):
canditate_list = [
[7, 5, 11], # elbow, shoulder, hip
[8, 6, 12],
[9, 7, 5], # wirst, elbow, shoulder
[10, 8, 6],
[5, 11, 13], # shoulder, hip, knee
[6, 12, 14],
[11, 13, 15], # hip, knee, angle
[12, 14, 16]
]
err_list = []
for p_idx in canditate_list:
if not self.angle_match(pose_ref, pose_cur, p_idx):
err_list.append(p_idx)
return err_list
At the end
This project requires local operation , Basically operational , The stability and matching effect of the program need to be further optimized .
More function upgrades will be made later , For example, try 3D Posture , Multiplayer mode ,Unity wait .
About author
- Head of Chengdu oar pilot group
- PPDE
- AICA Third stage students
I am here AI Studio Get diamond grade on , Lighten up 10 A badge , Let's talk to each other ~
https://aistudio.baidu.com/aistudio/personalcenter/thirdview/89442
边栏推荐
- [experience] install Visio on win11
- Yunxiaoduo software internal test distribution test platform description document
- Problems encountered in installing mysql8 on MAC
- 华为路由器如何配置静态路由
- [string] palindrome string of codeup
- 28io stream, byte output stream writes multiple bytes
- [experience] when ultralso makes a startup disk, there is an error: the disk / image capacity is too small
- 进程和线程
- 实践分享:如何安全快速地从 Centos迁移到openEuler
- Node 之 nvm 下载、安装、使用,以及node 、nrm 的相关使用
猜你喜欢
c语言——冒泡排序
B站刘二大人-线性回归 Pytorch
[cloud native] 3.1 kubernetes platform installation kubespher
ArcGIS application foundation 4 thematic map making
【经验】win11上安装visio
[SQL Server fast track] - authentication and establishment and management of user accounts
P2802 回家
Vulhub vulnerability recurrence 72_ uWSGI
Rustdesk builds its own remote desktop relay server
Problems encountered in installing mysql8 on MAC
随机推荐
Redis message queue
P2802 回家
无代码六月大事件|2022无代码探索者大会即将召开;AI增强型无代码工具推出...
01. Project introduction of blog development project
Node 之 nvm 下载、安装、使用,以及node 、nrm 的相关使用
数字经济破浪而来 ,LTD是权益独立的Web3.0网站?
备忘一下jvxetable的各种数据集获取方法
B站刘二大人-多元逻辑回归 Lecture 7
29io stream, byte output stream continue write line feed
Processes and threads
算法-- 爬楼梯(Kotlin)
H3C S5820V2_5830V2交换机IRF2堆叠后升级方法
[Jiudu OJ 08] simple search x
P2802 go home
How can large websites choose better virtual machine service providers?
wib3.0 跨越,在跨越(ง •̀_•́)ง
Remember an error in MySQL: the user specified as a definer ('mysql.infoschema '@' localhost ') does not exist
进程和线程
Game push image / table /cv/nlp, multi-threaded start
26file filter anonymous inner class and lambda optimization