๐
Mobile 2D Single Person (Or Your Own Object) Pose Estimation for TensorFlow 2.0
This repository is forked from edvardHua/PoseEstimationForMobile when the original repository was closed.
edvardHua/PoseEstimationForMobile repository is reopened! I'll maintain it separately.๐
This repository currently implemented the Hourglass model using TensorFlow 2.0 with Keras API.
Table of contents
- Goals
- Getting Started
- Results
- Converting To Mobile Model
- Tuning
- Details
- TODO
- Related Projects
- Acknowledgements
- Reference
- Contributing
- License
Goals
-
๐ Easy to train - ๐โ Easy to use the model on mobile device
Getting Started
Install Anaconda (~10 min)
Create Virtual Environment (~2 min)
Create new environment.
conda create -n {env_name} python={python_version} anaconda
# in my case
# conda create -n mpe-env-tf2-alpha0 python=3.7 anaconda
Start the environment.
source activate {env_name}
# in my case
# source activate mpe-env-tf2-alpha0
Install the requirements (~1 min)
cd {tf2-mobile-pose-estimation_path}
pip install -r requirements.txt
pip install git+https://github.com/philferriere/cocoapi.git@2929bd2ef6b451054755dfd7ceb09278f935f7ad#subdirectory=PythonAPI
Download original COCO dataset.
Download original COCO dataset
Special script that will help you to download and unpack needed COCO datasets. Please fill COCO_DATASET_PATH with path that is used in current version of repository. You can check needed path in file train.py
Warning Your system should have approximately 40gb of free space for datasets
python downloader.py --download-path=COCO_DATASET_PATH
Run The Project
In order to use the project you have to:
- Prepare the dataset(ai_challenger dataset) and unzip.
- Run the model using:
python train.py \
--dataset_config config/dataset/coco_single_person_only-gpu.cfg \
--experiment_config config/training/coco_single_experiment01-cpm-sg4-gpu.cfg
Compatiable Datasets
| Dataset Name | Doanload | Size | Number of images train/valid |
Number of Keypoints | Note |
|---|---|---|---|---|---|
| ai challenge | google drive | 2GB | 22k/1.5k | 14 | default dataset of this repo |
| coco single person only | google drive | 4GB | 25k/1k | 17 | filtered by showing only one person in an image which is from coco 2017 keypoint dataset |
- ai challenge's keypoint names:
['top_head', 'neck', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee', 'right_knee', 'left_ankle', 'right_ankle'] - coco's keypoint names:
['nose', 'left_eye', 'right_eye', 'left_ear', 'right_ear', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee', 'right_knee', 'left_ankle', 'right_ankle']
Results
AI Challenge Dataset
| Model Name | Backbone | Stage Or Depth | [email protected] | Size | Total Epoch | Total Training Time | Note |
|---|---|---|---|---|---|---|---|
| MobileNetV2 based CPM | cpm-b0 | Stage 1 | .. | .. | .. | .. | Default CPM |
| MobileNetV2 based CPM | cpm-b0 | Stage 2 | .. | .. | .. | .. | |
| MobileNetV2 based CPM | cpm-b0 | Stage 3 | .. | .. | .. | .. | |
| MobileNetV2 based CPM | cpm-b0 | Stage 4 | .. | .. | .. | .. | |
| MobileNetV2 based CPM | cpm-b0 | Stage 5 | .. | .. | .. | .. | |
| MobileNetV2 based Hourglass | hg-b0 | Depth 4 | .. | .. | .. | .. | Default Hourglass |
COCO Single persononly Dataset
| Model Name | Backbone | Stage Or Depth | OKS | Size | Total Epoch | Total Training Time | Note |
|---|---|---|---|---|---|---|---|
| MobileNetV2 based CPM | cpm-b0 | Stage 1 | .. | .. | .. | .. | Default CPM |
| MobileNetV2 based CPM | cpm-b0 | Stage 2 | .. | .. | .. | .. | |
| MobileNetV2 based CPM | cpm-b0 | Stage 3 | .. | .. | .. | .. | |
| MobileNetV2 based CPM | cpm-b0 | Stage 4 | .. | .. | .. | .. | |
| MobileNetV2 based CPM | cpm-b0 | Stage 5 | .. | .. | .. | .. | |
| MobileNetV2 based Hourglass | hg-b0 | Depth 4 | .. | .. | .. | .. | Default Hourglass |
Converting To Mobile Model
TensorFLow Lite
If you train the model, it will create tflite models per evaluation step.
Core ML
Check convert_to_coreml.py script. The converted .mlmodel support iOS14+.
Details
This section will be separated to other
.mdfile.
Folder Structure
tf2-mobile-pose-estimation
โโโ config
| โโโ model_config.py
| โโโ train_config.py
โโโ data_loader
| โโโ data_loader.py
| โโโ dataset_augment.py
| โโโ dataset_prepare.py
| โโโ pose_image_processor.py
โโโ models
| โโโ common.py
| โโโ mobilenet.py
| โโโ mobilenetv2.py
| โโโ mobilenetv3.py
| โโโ resnet.py
| โโโ resneta.py
| โโโ resnetd.py
| โโโ senet.py
| โโโ simplepose_coco.py
| โโโ simpleposemobile_coco.py
โโโ train.py - the main training script
โโโ common.py
โโโ requirements.txt
โโโ outputs - this folder will be generated automatically when start training
โโโ 20200312-sp-ai_challenger
| โโโ saved_model
| โโโ image_results
โโโ 20200312-sp-ai_challenger
โโโ ...
My SSD
โโโ datasets - this folder contains the datasets of the project.
โโโ ai_challenger
โโโ train.json
โโโ valid.json
โโโ train
โโโ valid
TODO
Save model to saved_modelConvert the model(saved_model) to TFLite model(.tflite)Convert the model(saved_model) to Core ML model(.mlmodel)Run the model on iOS- Release 1.0 models
- Support distributed GPUs training
- Make DEMO gif running on mobile device
- Run the model on Android
Reference
[1] Paper of Convolutional Pose Machines
[2] Paper of Stack Hourglass
[3] Paper of MobileNet V2
[4] Repository PoseEstimation-CoreML
[5] Repository of tf-pose-estimation
[6] Devlope guide of TensorFlow Lite
[7] Mace documentation
Related Projects
- tucan9389/PoseEstimation-CoreML
- tucan9389/PoseEstimation-TFLiteSwift (Preparing...)
- tucan9389/KeypointAnnotation
- osmr/imgclsmob
- edvardHua/PoseEstimationForMobile
- jwkanggist/tf-tiny-pose-estimation
- dongseokYang/Body-Pose-Estimation-Android-gpu
Other Pose Estimation Projects
Contributing
This section will be separated to other
.mdfile.
Any contributions are welcome including improving the project.
![[PR] Support lightweight cpm and hourglass](https://avatars.githubusercontent.com/u/37643248?v=4)
1 Aug 22, 2022
275 Dec 28, 2022
6 May 13, 2022
7 Dec 26, 2022
108 Dec 31, 2022
9 Nov 04, 2022
3.9k Dec 23, 2022
294 Dec 12, 2022
249 Dec 28, 2022
214 Dec 28, 2022
193 Dec 05, 2022
2.1k Dec 27, 2022
23 Oct 25, 2022
350 Dec 28, 2022
61 Nov 25, 2022
60 Dec 16, 2022
116 Nov 24, 2022
721 Jan 03, 2023
21 Sep 30, 2022
3 Feb 27, 2022