Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A Python library for rendering ASS subtitle file format using libass.

ass_renderer A Python library for rendering ASS subtitle file format using libass. Installation pip install --user ass-renderer Contributing # Clone

1 Nov 02, 2022
Bearer API client for Python

Bearer Python Bearer Python client Installation pip install bearer Usage Get your Bearer Secret Key and integration id from the Dashboard and use the

Bearer 9 Oct 31, 2022
Python library for generating sequences with uniform stimulus history

Sampling Euler tours for uniform stimulus history Table of Contents About Examples Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Co

5 Nov 11, 2021
Grade Notifyer Bot

A bot that automatically crawl the submission platform of montefiore to notify the student when a project has been graded.

Julien Gustin 2 Jun 02, 2022
A Telegram bot to download posts, videos, reels, IGTV and a user profile picture from Instagram!

Telegram Bot A telegram bot to download media from Instagram! No API Key or Login Needed! Requirements You must have python installed (of course) You

Simon Farah 2 Apr 10, 2022
Moon-TikTok-Checker - A TikTok Username checking tool that probably 3/4 people use to get rare usernames

Moon Checker (educational Purposes Only) What Is Moon Checker? This is a TikTok

glide 4 Nov 30, 2022
0-1 knapsack with an additional constraint of maximum number of items used

extended_knapsack 0-1 knapsack with an additional constraint of maximum number of items used. Usage pip3 install extended_knapsack from extended_knaps

Lion Ralfs 1 Jan 06, 2022
A tool to build scripts to toggle between minimal & default services in Windows based on user defined lists.

A tool to build scripts to toggle between minimal & default services in Windows based on user defined lists.

AMIT 29 Jan 01, 2023
ANKIT-OS/STYLISH-TEXT is a special repository. Its Is A Telegram Bot Which Can Translate Your Text Into 100+ Language

🔥 ᴳᴼᴼᴳᴸᴱ⁻ᵀᴿᴬᴺᔆᴸᴬᵀᴱᴿ 🔥 The owner would not be responsible for any kind of bans due to the bot. • ⚡ INSTALLING ⚡ • • 🛠️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs 🔰 • If

ANKIT KUMAR 1 Dec 23, 2021
A powerful discord bot for forming team.

Discord_SquadBot A powerful discord bot for forming team. Pre-requirement Python 3.7 and latest Discord.py module is required. Installation guideline

Jacky Yu 2 Jan 29, 2022
Python lib to control HottoH based stove devices

Project desciption This library can be used to discuss with HootoH based stove devices Actually tested and validated with a CMG Drum stove. To use thi

3 May 16, 2022
A Discord bot that automatically saves SHSH blobs for all of your iOS devices.

AutoTSS AutoTSS is a Discord bot that automatically saves SHSH blobs for all of your iOS devices. Want a CLI automatic blob saver? Check out AutoTSS-c

adam 79 Dec 13, 2022
Compulsory join Telegram Bot

mussjoin About Compulsory join Telegram Bot this Telegram Bot Application can be added users to Telegram Channel or Group compulsorily. in addition wh

Hamed Mohammadvand 4 Dec 03, 2021
Automation application was made by me using Google, Sheet and Slack APIs with Python.

README This application is used to transfer the data in the xlsx document we have to the Google Drive environment and calculate the "total budget" wit

3 Apr 12, 2022
python based bot Sends notification to your telegram whenever a new video is released on a youtube channel!

YTnotifier python based bot Sends notification to your telegram whenever a new video is released on a youtube channel! REQUIREMENTS telethon python-de

Mohamed Rizad 6 Jul 23, 2022
Uma API pública contendo informações sobre o unvierso de Roberto Gomez Bolaños.

Chespirito API Objetivo Esta API tem como objetivo ser um ponto de referência para a procura sobre todo o universo do grande Roberto Gomez Bolaños, ta

Pery Lemke 6 Feb 02, 2022
A Python wrapper around the Pushbullet API to send different types of push notifications to your phone or/and computer.

pushbullet-python A Python wrapper around the Pushbullet API to send different types of push notifications to your phone or/and computer. Installation

Janu Lingeswaran 1 Jan 14, 2022
Example app to be deployed to AWS as an API Gateway / Lambda Stack

Disclaimer I won't answer issues or emails regarding the project anymore. The project is old and not maintained anymore. I'm not sure if it still work

Ben 123 Jan 01, 2023
A Bot that Forwards Tweets to Telegram using Airtable as a database.

Twitter Telegram Forward A Bot that Forwards Tweets to Telegram using Airtable as a Database. Features: Handles multiple twitter and telegram channels

George Bakev 3 Dec 21, 2022
A Telegram user bot to count telegram channel subscriber or group member.

Subscriber Count Userbot A Telegram user bot to count telegram channel subscriber or group member. This tool is only for educational purpose. You coul

IDNCoderX 8 Nov 30, 2022