OpenMMLab Pose Estimation Toolbox and Benchmark.

Last update: Dec 31, 2022

Overview

Introduction

English | 简体中文

MMPose is an open-source toolbox for pose estimation based on PyTorch. It is a part of the OpenMMLab project.

The master branch works with PyTorch 1.3+.

COCO 17-keypoint pose estimation

133-keypoint whole-body pose estimation (full HD version)

2D animal_pose estimation

Major Features

Support diverse tasks

We support a wide spectrum of mainstream pose analysis tasks in current research community, including 2d multi-person human pose estimation, 2d hand pose estimation, 2d face landmark detection, 133 keypoint whole-body human pose estimation, 3d human mesh recovery, fashion landmark detection and animal pose estimation. See demo.md for more information.
Higher efficiency and higher accuracy

MMPose implements multiple state-of-the-art (SOTA) deep learning models, including both top-down & bottom-up approaches. We achieve faster training speed and higher accuracy than other popular codebases, such as HRNet. See benchmark.md for more information.
Support for various datasets

The toolbox directly supports multiple popular and representative datasets, COCO, AIC, MPII, MPII-TRB, OCHuman etc. See data_preparation.md for more information.
Well designed, tested and documented

We decompose MMPose into different components and one can easily construct a customized pose estimation framework by combining different modules. We provide detailed documentation and API reference, as well as unittests.

Model Zoo

Supported algorithms:

(click to collapse)

Supported datasets:

(click to collapse)

Supported backbones:

(click to expand)

Results and models are available in the README.md of each method's config directory. A summary can be found in the model zoo page. We will keep up with the latest progress of the community, and support more popular algorithms and frameworks.

If you have any feature requests, please feel free to leave a comment in Issues.

Benchmark

We demonstrate the superiority of our MMPose framework in terms of speed and accuracy on the standard COCO keypoint detection benchmark.

Model	Input size	MMPose (s/iter)	HRNet (s/iter)	MMPose (mAP)	HRNet (mAP)
resnet_50	256x192	0.28	0.64	0.718	0.704
resnet_50	384x288	0.81	1.24	0.731	0.722
resnet_101	256x192	0.36	0.84	0.726	0.714
resnet_101	384x288	0.79	1.53	0.748	0.736
resnet_152	256x192	0.49	1.00	0.735	0.720
resnet_152	384x288	0.96	1.65	0.750	0.743
hrnet_w32	256x192	0.54	1.31	0.746	0.744
hrnet_w32	384x288	0.76	2.00	0.760	0.758
hrnet_w48	256x192	0.66	1.55	0.756	0.751
hrnet_w48	384x288	1.23	2.20	0.767	0.763

More details about the benchmark are available on benchmark.md.

Installation

Please refer to install.md for installation.

Data Preparation

Please refer to data_preparation.md for a general knowledge of data preparation.

Get Started

Please see getting_started.md for the basic usage of MMPose. There are also tutorials:

FAQ

Please refer to FAQ for frequently asked questions.

License

This project is released under the Apache 2.0 license.

Citation

If you find this project useful in your research, please consider cite:

@misc{mmpose2020,
    title={OpenMMLab Pose Estimation Toolbox and Benchmark},
    author={MMPose Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmpose}},
    year={2020}
}

Contributing

We appreciate all contributions to improve MMPose. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMPose is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new models.

Projects in OpenMMLab

MMCV: OpenMMLab foundational library for computer vision.
MMClassification: OpenMMLab image classification toolbox and benchmark.
MMDetection: OpenMMLab detection toolbox and benchmark.
MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
MMTracking: OpenMMLab video perception toolbox and benchmark.
MMPose: OpenMMLab pose estimation toolbox and benchmark.
MMEditing: OpenMMLab image and video editing toolbox.
MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding.

Comments

Can noise be added to Dataset.

Hallo Team,

I was training the hrnet model and trying to improve the accuracy of model since trained the model too many times and it may lead to overfitting.

I would like to know if there is a possibility to augment the data with random noise in MMPOSE?

Where to look into the code of mmpose and how we can do this ?

Please suggest !
kind/discussion

opened by YuktiADY 68
How to Test on images as well as synthetic images

I have trained model on my custom dataset and the results are pretty much good **Best AP is 0.9624 at 30 epoch..

How ca i test some images or some synthetic images on my trained model ??
question

opened by YuktiADY 64
COCOeval' object has no attribute 'score_key'

Hello,

While training i am getting AttributeError: 'COCOeval' object has no attribute 'score_key'

Traceback (most recent call last): File "./mmpose/tools/train.py", line 170, in main() File "./mmpose/tools/train.py", line 166, in main meta=meta) File "/home/yukti/mmpose/mmpose/mmpose/apis/train.py", line 192, in train_model runner.run(data_loaders, cfg.workflow, cfg.total_epochs) File "/home/yukti/mmpose/mmcv/mmcv/runner/epoch_based_runner.py", line 127, in run epoch_runner(data_loaders[i], **kwargs) File "/home/yukti/mmpose/mmcv/mmcv/runner/epoch_based_runner.py", line 54, in train self.call_hook('after_train_epoch') File "/home/yukti/mmpose/mmcv/mmcv/runner/base_runner.py", line 309, in call_hook getattr(hook, fn_name)(self) File "/home/yukti/mmpose/mmcv/mmcv/runner/hooks/evaluation.py", line 267, in after_train_epoch self._do_evaluate(runner) File "/home/yukti/mmpose/mmcv/mmcv/runner/hooks/evaluation.py", line 505, in _do_evaluate key_score = self.evaluate(runner, results) File "/home/yukti/mmpose/mmpose/mmpose/core/evaluation/eval_hooks.py", line 139, in evaluate **self.eval_kwargs) File "/home/yukti/mmpose/mmpose/mmpose/datasets/datasets/top_down/topdown_coco_dataset.py", line 317, in evaluate info_str = self._do_python_keypoint_eval(res_file) File "/home/yukti/mmpose/mmpose/mmpose/datasets/datasets/top_down/topdown_coco_dataset.py", line 371, in _do_python_keypoint_eval coco_eval.evaluate() File "/home/yukti/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/xtcocotools/cocoeval.py", line 257, in evaluate for imgId in p.imgIds File "/home/yukti/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/xtcocotools/cocoeval.py", line 258, in for catId in catIds} File "/home/yukti/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/xtcocotools/cocoeval.py", line 311, in computeOks inds = np.argsort([-d[self.score_key] for d in dts], kind='mergesort') File "/home/yukti/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/xtcocotools/cocoeval.py", line 311, in inds = np.argsort([-d[self.score_key] for d in dts], kind='mergesort') AttributeError: 'COCOeval' object has no attribute 'score_key'

opened by YuktiADY 57
prediction and see the results

Hi I see in the documentation that we can do prediction on the complete dataset and generate a JSON of predictions or see the ground truth keypoints of the dataset but I see nowhere how to do the prediction on a new image and to see the predictions on the image predicted. The JSON file format generated by prediction is incompatible with those of the visualization. Furthermore I launched my training and after 140 epochs I have a loss of 2.0 and my keypoints are far of where it would be, the max score I have is 0.35. How to improve it please ?

opened by SylvainArd 43
Questions about the polygon annotation in the dataset

Hello,. I would like to know if the points in the JSON training file formatted in COCO must be inside their polygons or if it does not matter. Moreover I would like to know if the detection of points takes into account the neighborhood of the point on the one hand and the neighborhood of its polygon on the other hand. Finally I would like to know how to customize my point and polygon classes in your software. Thanks Cordially

opened by SylvainArd 36
RESUME FROM in the config doesnt work

Hallo ,

I already trained hrnet[w32]_256x192 model and now i want to again continue training for some epochs so i gave in the config. resume_from = '/home/yukti/mmpose/theodore_2022-04-25/best_AP_epoch_30.pth'

but the training is not resuming from this epoch rather ti shows this .

2022-05-03 14:47:57,328 - mmpose - INFO - workflow: [('train', 1)], max: 30 epochs 2022-05-03 14:47:57,328 - mmpose - INFO - Checkpoints will be saved to /home/yukti/mmpose/theodore_2022-05-03 by HardDiskBackend. INFO:torch.distributed.elastic.agent.server.api:[default] worker group successfully finished. Waiting 300 seconds for other agents to finish. INFO:torch.distributed.elastic.agent.server.api:Local worker group finished (SUCCEEDED). Waiting 300 seconds for other agents to finish /home/yukti/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/distributed/elastic/utils/store.py:71: FutureWarning: This is an experimental API and will be changed in future. "This is an experimental API and will be changed in future.", FutureWarning INFO:torch.distributed.elastic.agent.server.api:Done waiting for other agents. Elapsed: 0.0006282329559326172 seconds {"name": "torchelastic.worker.status.SUCCEEDED", "source": "WORKER", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": 0, "group_rank": 0, "worker_id": "3529801", "role": "default", "hostname": "dst-toaster.etit.tu-chemnitz.de", "state": "SUCCEEDED", "total_run_time": 45, "rdzv_backend": "static", "raw_error": null, "metadata": "{"group_world_size": 1, "entry_point": "python", "local_rank": [0], "role_rank": [0], "role_world_size": [1]}", "agent_restarts": 0}} {"name": "torchelastic.worker.status.SUCCEEDED", "source": "AGENT", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": null, "group_rank": 0, "worker_id": null, "role": "default", "hostname": "dst-toaster.etit.tu-chemnitz.de", "state": "SUCCEEDED", "total_run_time": 45, "rdzv_backend": "static", "raw_error": null, "metadata": "{"group_world_size": 1, "entry_point": "python"}", "agent_restarts": 0}}

Last time ti worked but not this time. Please suggest!

opened by YuktiADY 26
ValueError: Unknown projection '3d'

由于没有路径文件的关系，自己上传了一个文件运行时显示：RuntimeError: Only one file(not dir) is allowed in the zipfile

自己预处理把model下载，定义了下输入文件和输出文件路径后发生如下报错，请问该怎样做才能复现demo呢 Use load_from_local loader [ ] 0/719, elapsed: 0s, ETA:Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/matplotlib/projections/init.py", line 58, in get_projection_class return projection_registry.get_projection_class(projection) File "/usr/local/lib/python3.6/dist-packages/matplotlib/projections/init.py", line 25, in get_projection_class return self._all_projection_types[name] KeyError: '3d'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "test.py", line 333, in main() File "test.py", line 317, in main num_instances=num_instances) File "/usr/local/lib/python3.6/dist-packages/mmpose/apis/inference_3d.py", line 423, in vis_3d_pose_result out_file=out_file) File "/usr/local/lib/python3.6/dist-packages/mmpose/models/detectors/pose_lifter.py", line 388, in show_result num_instances=num_instances) File "/usr/local/lib/python3.6/dist-packages/mmpose/core/visualization/image.py", line 274, in imshow_keypoints_3d ax = fig.add_subplot(1, num_axis, ax_idx, projection='3d') File "/usr/local/lib/python3.6/dist-packages/matplotlib/figure.py", line 1396, in add_subplot self._process_projection_requirements(*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/matplotlib/figure.py", line 1120, in _process_projection_requirements projection_class = projections.get_projection_class(projection) File "/usr/local/lib/python3.6/dist-packages/matplotlib/projections/init.py", line 60, in get_projection_class raise ValueError("Unknown projection %r" % projection) ValueError: Unknown projection '3d'

opened by 642802017 23
how to decode onnx model and get final results on image?

After run pytorch2onnx.py, I have got the onnx model, but I have not found the relevant documents that can infer it to check the correctness of resultes of the onnx model. How can I do it? Or do you have the relevant reasoning scripts for reference ONNX MODEL? I need your help
kind/discussion onnx

opened by zcc720 22
UserWarning about mmpose1.0

When I run runner.val , it show :"UserWarning: Since the spatial dimensions of overlaid_image: (488, 640) and featmap: torch.Size([64, 48]) are not same, the feature map will be interpolated. This may cause mismatch problems ！" waht's my problem
info/1.x

opened by ChenZhenGui 19
[Enhance] Speedup the Video Inference by Accelerating data-loading Stage in Top_Down_Full_Frame_without_Det
It's a parallel to my .

Motivation

The video inference was not efficient, because there are many "resize", "padding", "to rgb" and "normalize" operations in CPU workload. Besides, the 'img_metas' is calculated for every frame, which is nonefficient. The CPU workload is really high, but the speed is slow.

Modification

I transformed the video "crop", "resize", "padding", "to rgb" in ffmpeg-based video reader, which is lite and CPU friendly. The video reader can support NVIDIA-VIDEO-DECODING if possible, to save more CPU workload. And the "normalize" is done in GPU. In all, I reduced the data-loading time and CPU workload.

Result

For the pure data loading, it can reach 400 Fps (possible the ceiling of NVIDIA-DECODER). The CPU work load is acceptable (450%).

for frame_resize, frame_origin in zip( mmcv.track_iter_progress(video_resize), video_origin): # test a single frame without model inference data = process_img(frame_resize, img_metas, args.device) continue pose_results = pose_model( return_loss=False, return_heatmap=False, **data)

The video size is 1280x800, converting to image size 512x512.

Limitation

Only works for top_down_video_demo_full_frame_without_det.py. May be adaptive to bottom_up_video_demo.py.

Didn't support for flip-pair test.

Checklist

Before PR:

[x] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.

[x] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.

After PR:

[x] CLA has been signed and all committers have signed the CLA in this PR.
opened by chenxinfeng4 18
Low COCO Evaluation results although acc_pose is high

Hi, I am trying to implement a new data augmentation technique into the train pipeline. I have incorporated the technique after TopDownAffine in the train pipeline. When I start training the model, I get good acc_pose values and the loss also decreases however when the evaluation is done (after 50 epochs), the resulting metrics are very poor:

Average Precision (AP) @[ IoU=0.50:0.95 | type= all | maxDets= 20 ] = 0.001 Average Precision (AP) @[ IoU=0.50 | type= all | maxDets= 20 ] = 0.009 Average Precision (AP) @[ IoU=0.75 | type= all | maxDets= 20 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | type=medium | maxDets= 20 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | type= large | maxDets= 20 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | type= all | maxDets= 20 ] = 0.015 Average Recall (AR) @[ IoU=0.50 | type= all | maxDets= 20 ] = 0.087 Average Recall (AR) @[ IoU=0.75 | type= all | maxDets= 20 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | type=medium | maxDets= 20 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | type= large | maxDets= 20 ] = 0.017

What can be the possible reason for this? Should I incorporate the augmentation method in the val pipeline as well? The method simply merges the image with its mask image and returns the result.
question

opened by rubeea 18
[Fix] Update minimum python version to 3.7
Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

[ ] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.

[ ] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.

[ ] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.

[ ] New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.

[ ] The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

[ ] CLA has been signed and all committers have signed the CLA in this PR.
opened by Ben-Louis 1

mmpose1.x NOT work for training my own dataset

I follow the repo and train the demo loucst dataset. It works and here is the training log:

01/07 10:54:26 - mmengine - INFO - Epoch(train) [1][50/79]  lr: 4.954910e-05  eta: 0:02:59  time: 0.241931  data_time: 0.098693  memory: 1397  loss: 0.006913  loss_kpt: 0.006913  acc_pose: 0.178571
01/07 10:54:29 - mmengine - INFO - Exp name: td-hm_res101_8xb64-210e_locust-160x160_20230107_105408
01/07 10:54:29 - mmengine - INFO - Saving checkpoint at 1 epochs
01/07 10:54:36 - mmengine - INFO - Epoch(train) [2][50/79]  lr: 1.286283e-04  eta: 0:01:26  time: 0.096829  data_time: 0.001756  memory: 1396  loss: 0.004748  loss_kpt: 0.004748  acc_pose: 0.407143
01/07 10:54:39 - mmengine - INFO - Exp name: td-hm_res101_8xb64-210e_locust-160x160_20230107_105408
01/07 10:54:39 - mmengine - INFO - Saving checkpoint at 2 epochs

and Then I want to use my own dataset. It has been transformed to coco format. Follow locust dataset, similarly，I get the training log below:

01/07 11:33:50 - mmengine - INFO - Checkpoints will be saved to C:\Work\Proj\PyProj\mmpose-1.x\mmpose-1.x\tools\work_dirs\td-hm_res101_8xb64-210e_chicken-160x160.
01/07 11:34:07 - mmengine - INFO - Epoch(train) [1][50/204]  lr: 4.954910e-05  eta: 0:11:45  time: 0.354480  data_time: 0.154573  memory: 2239  loss: 0.000000  loss_kpt: 0.000000  acc_pose: 0.000000
01/07 11:34:18 - mmengine - INFO - Epoch(train) [1][100/204]  lr: 9.959920e-05  eta: 0:09:13  time: 0.216220  data_time: 0.065922  memory: 2239  loss: 0.000003  loss_kpt: 0.000003  acc_pose: 0.000000
01/07 11:34:29 - mmengine - INFO - Epoch(train) [1][150/204]  lr: 1.496493e-04  eta: 0:08:14  time: 0.213670  data_time: 0.064010  memory: 2239  loss: 0.000000  loss_kpt: 0.000000  acc_pose: 0.000000
01/07 11:34:39 - mmengine - INFO - Epoch(train) [1][200/204]  lr: 1.996994e-04  eta: 0:07:36  time: 0.207582  data_time: 0.058121  memory: 2239  loss: 0.000000  loss_kpt: 0.000000  acc_pose: 0.000000
01/07 11:34:40 - mmengine - INFO - Exp name: td-hm_res101_8xb64-210e_chicken-160x160_20230107_113344
01/07 11:34:40 - mmengine - INFO - Saving checkpoint at 1 epochs
01/07 11:34:54 - mmengine - INFO - Epoch(train) [2][50/204]  lr: 2.537535e-04  eta: 0:06:57  time: 0.194315  data_time: 0.037169  memory: 2239  loss: 0.000000  loss_kpt: 0.000000  acc_pose: 0.000000
01/07 11:35:04 - mmengine - INFO - Epoch(train) [2][100/204]  lr: 3.038036e-04  eta: 0:06:32  time: 0.188513  data_time: 0.035870  memory: 2239  loss: 0.000000  loss_kpt: 0.000000  acc_pose: 0.000000
01/07 11:35:13 - mmengine - INFO - Epoch(train) [2][150/204]  lr: 3.538537e-04  eta: 0:06:13  time: 0.193783  data_time: 0.044105  memory: 2239  loss: 0.000000  loss_kpt: 0.000000  acc_pose: 0.000000

the loss 、loss_kpt and acc_pose are always ZERO.

Here are some details:

(1) config file:

_base_ = ['../../../_base_/default_runtime.py' ]

# runtime
train_cfg = dict(max_epochs=10, val_interval=10) #210

# optimizer
optim_wrapper = dict(optimizer=dict(
    type='Adam',
    lr=5e-4,
))

# learning policy
param_scheduler = [
    dict(
        type='LinearLR', begin=0, end=500, start_factor=0.001,
        by_epoch=False),  # warm-up
    dict(
        type='MultiStepLR',
        begin=0,
        end=210,
        milestones=[170, 200],
        gamma=0.1,
        by_epoch=True)
]

# automatically scaling LR based on the actual training batch size
auto_scale_lr = dict(base_batch_size=512)

# hooks
default_hooks = dict(checkpoint=dict(save_best='coco/AP', rule='greater'))

# codec settings
codec = dict(
    type='MSRAHeatmap', input_size=(160, 160), heatmap_size=(40, 40), sigma=2)

# model settings
model = dict(
    type='TopdownPoseEstimator',
    data_preprocessor=dict(
        type='PoseDataPreprocessor',
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        bgr_to_rgb=True),
    backbone=dict(
        type='ResNet',
        depth=101,
        init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet101'),
    ),
    head=dict(
        type='HeatmapHead',
        in_channels=2048,
        out_channels=10, #change 35 to 10
        loss=dict(type='KeypointMSELoss', use_target_weight=True),
        decoder=codec),
    test_cfg=dict(
        flip_test=True,
        flip_mode='heatmap',
        shift_heatmap=True,
    ))

# base dataset settings
dataset_type = 'ChickenDataset'
data_mode = 'topdown'
data_root = '../data/chicken/'


# pipelines
train_pipeline = [
    dict(type='LoadImage', file_client_args={{_base_.file_client_args}}),
    dict(type='GetBBoxCenterScale', padding=0.8),
    dict(type='RandomFlip', direction='horizontal'),
    dict(
        type='RandomBBoxTransform',
        shift_factor=0.25,
        rotate_factor=180,
        scale_factor=(0.7, 1.3)),
    dict(type='TopdownAffine', input_size=codec['input_size']),
    dict(type='GenerateTarget', target_type='heatmap', encoder=codec),
    dict(type='PackPoseInputs')
]
val_pipeline = [
    dict(type='LoadImage', file_client_args={{_base_.file_client_args}}),
    dict(type='GetBBoxCenterScale', padding=0.8),
    dict(type='TopdownAffine', input_size=codec['input_size']),
    dict(type='PackPoseInputs')
]

# data loaders
train_dataloader = dict(
    batch_size=8, #64
    num_workers=2,
    persistent_workers=True,
    sampler=dict(type='DefaultSampler', shuffle=True),
    dataset=dict(
        type=dataset_type,
        data_root=data_root,
        data_mode=data_mode,
        ann_file='annotations/train.json',
        data_prefix=dict(img='images/'),
        pipeline=train_pipeline,
    ))
val_dataloader = dict(
    batch_size=32,
    num_workers=2,
    persistent_workers=True,
    drop_last=False,
    sampler=dict(type='DefaultSampler', shuffle=False, round_up=False),
    dataset=dict(
        type=dataset_type,
        data_root=data_root,
        data_mode=data_mode,
        ann_file='annotations/val.json',
        data_prefix=dict(img='images/'),
        test_mode=True,
        pipeline=val_pipeline,
    ))
test_dataloader = val_dataloader

# evaluators
val_evaluator = [
    dict(type='PCKAccuracy', thr=0.2),
    dict(type='AUC'),
    dict(type='EPE'),
]
test_evaluator = val_evaluator

(2) ann files:

"images": [
        {
            "height": 1920,
            "width": 2560,
            "id": 1,
            "dir_name": "train",
            "file_name": "ch02_20210212124211_001046.jpg"
        },
        {
            "height": 1920,
            "width": 2560,
            "id": 2,
            "dir_name": "train",
            "file_name": "ch02_20210213143154_000524.jpg"


        ...

"image_id": 922,
            "bbox": [
                762,
                631,
                1415,
                1288
            ],
            "category_id": 1,
            "id": 1545
        },
        {
            "segmentation": [
                0
            ],
            "num_keypoints": 9,
            "area": 1807656,
            "iscrowd": 0,
            "keypoints": [
                715,
                1125,
                2,
                0,
                0,
                0,
                786,
                1584,
                2,
                699,
                1568,
                2,
                799,
                1746,
                2,
                643,
                1703,
                2,
                392,
                623,
                2,
                299,
                637,
                2,
                354,
                595,
                2,
                343,
                697,
                2
            ],
    ...

（3）point file

dataset_info = dict(
    dataset_name='chicken',
    paper_info=dict(
        author='yuan ',
        title='chicken',
        container='',
        year='2022',
        homepage='',
    ),
    keypoint_info={
        0:
        dict(name='body_center', id=0, color=[51, 153, 255], type='', swap=''),
        1:
        dict(
            name='body_tail',
            id=1,
            color=[51, 153, 255],
            type='',
            swap=''),
        2:
        dict(
            name='body_knee_left',
            id=2,
            color=[51, 153, 255],
            type='',
            swap=''),
        3:
        dict(
            name='body_knee_right',
            id=3,
            color=[51, 153, 255],
            type='',
            swap=''),
        4:
        dict(
            name='body_heel_left',
            id=4,
            color=[51, 153, 255],
            type='',
            swap=''),
        5:
        dict(
            name='body_heel_right',
            id=5,
            color=[0, 255, 0],
            type='',
            swap=''),
        6:
        dict(
            name='eye_left',
            id=6,
            color=[255, 128, 0],
            type='',
            swap=''),
        7:
        dict(
            name='eye_right',
            id=7,
            color=[0, 255, 0],
            type='',
            swap=''),
        8:
        dict(
            name='comb',
            id=8,
            color=[255, 128, 0],
            type='',
            swap=''),
        9:
        dict(
            name='beak',
            id=9,
            color=[0, 255, 0],
            type='',
            swap=''),
    },
    skeleton_info={
        0:
        dict(link=('body_center', 'body_tail'), id=0, color=[0, 255, 0]),
        1:
        dict(link=('body_center', 'body_knee_left'), id=1, color=[0, 255, 0]),
        2:
        dict(link=('body_center', 'body_knee_right'), id=2, color=[255, 128, 0]),
        3:
        dict(link=('body_center', 'eye_left'), id=3, color=[255, 128, 0]),
        4:
        dict(link=('body_knee_left', 'body_heel_left'), id=4, color=[51, 153, 255]),
        5:
        dict(link=('body_knee_right', 'body_heel_right'), id=5, color=[0, 255, 0]),
        6:
        dict(
            link=('eye_left', 'comb'), id=6, color=[255, 128, 0]),
        7:
        dict(link=('eye_left', 'beak'), id=7, color=[0, 255, 0]),
    },
    joint_weights=[
        1., 1., 1., 1., 1., 1., 1., 1., 1.2, 1.2
    ],
    sigmas=[
        0.026, 0.025, 0.025, 0.035, 0.035, 0.029, 0.029, 0.072, 0.079, 0.079
    ]
)

Maybe (1) my image is too big? 2560 * 1920? (2) in config file, input_size = (160,160) is not suitable？

Anthing will be appreciated, Thanks.

opened by github2016-yuan 1

MMPose_Tutorial.ipynb error in a particular cell

the below command

"from mmpose.apis import inference_top_down_pose_model,init_pose_model,vis_pose_result, process_mmdet_results"

throws error.

PFB error details. /usr/local/lib/python3.8/dist-packages/xtcocotools/mask.py in 1 author = 'tsungyi' 2 ----> 3 import xtcocotools._mask as _mask 4 5 # Interface for manipulating masks stored in RLE format.

xtcocotools/_mask.pyx in init xtcocotools._mask()

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

I tried to upgrade numpy as well, but it did not work.

opened by SoumyajitMukherjee-droid 1
MIm command not working

when i try to run "!mim download mmpose --config associative_embedding_hrnet_w32_coco_512x512 --dest ." in a cell in Jupyter notebook, it gives the following error.

"'mim' is not recognized as an internal or external command, operable program or batch file."

i installed openmim "pip install openmim" it got successfully installed, and i have restarted the kernel as well, then also it throws the above error

opened by SoumyajitMukherjee-droid 0
available output_layer_names in demo

How can I find out which layers are available for any network to define here - https://github.com/open-mmlab/mmpose/blob/1f11d6ccbaa034e50847feb92bc7cf7760a2e387/demo/top_down_video_demo_with_mmdet.py#L138 ?

i would specifically like to log the bottleneck, which is not available for a ResNet for some reason?

opened by iulsko 1

Releases(v1.0.0rc0)

v1.0.0rc0(Oct 14, 2022)
New Features

Support 4 light-weight pose estimation algorithms: SimCC (ECCV'2022), Debias-IPR (ICCV'2021), IPR (ECCV'2018), and DSNT (ArXiv'2018) (#1628) @Tau-J

Migrations

Add Webcam API in MMPose 1.0 (#1638, #1662) @Ben-Louis

Add codec for Associative Embedding (beta) (#1603) @ly015

Improvements

Add a colab tutorial for MMPose 1.0 (#1660) @Tau-J

Add model index in config folder (#1710, #1709, #1627) @ly015, @Tau-J, @Ben-Louis

Update and improve documentation (#1692, #1656, #1681, #1677, #1664, #1659) @Tau-J, @Ben-Louis, @liqikai9

Improve config structures and formats (#1651) @liqikai9

Bug Fixes

Update mmengine version requirements (#1715) @Ben-Louis

Update dependencies of pre-commit hooks (#1705) @Ben-Louis

Fix mmcv version in DockerFile (#1704)

Fix a bug in setting dataset metainfo in configs (#1684) @ly015

Fix a bug in UDP training (#1682) @liqikai9

Fix a bug in Dark decoding (#1676) @liqikai9

Fix bugs in visualization (#1671, #1668, #1657) @liqikai9, @Ben-Louis

Fix incorrect flops calculation (#1669) @liqikai9

Fix tensor.tile compatibility issue for pytorch 1.6 (#1658) @ly015

Fix compatibility with MultilevelPixelData (#1647) @liqikai9

Source code(tar.gz)
Source code(zip)
v0.29.0(Oct 14, 2022)
Highlights

Support DEKR, CVPR'2021 (#1693) @JinluZhang1126, @jin-s13, @Ben-Louis

Support CID, CVPR'2022 (#1604) @kennethwdk, @jin-s13, @Ben-Louis

Improvements

Improve 3D pose estimation demo with viewpoint & height controlling and enriched input formats (#1481, #1490) @pallgeuer, @liqikai9

Improve smoothing of 3D video pose estimation (#1479) @darcula1993

Improve dependency management for smoother installation (#1491) @danielbraun89

Improve documentation quality (#1680, #1525, #1516, #1513, #1511, #1507, #1506, #1525) @Ben-Louis, @Yulv-git, @jin-s13

Bug Fixes

Update dependency versions of pre-commit-hooks (#1706, #1678) @Ben-Louis, @liqikai9

Fix missing data transforms in RLE configs (#1632) @Ben-Louis

Fix a bug in fliplr_joints that causes error when keypoint visibility has float values (#1589) @walsvid

Fix hsigmoid default parameters (#1575) @Ben-Louis

Fix a bug in UDP decoding (#1565) @liqikai9

Fix a bug in pose tracking demo with non-COCO dataset (#1504) @daixinghome

Fix typos and unused contents in configs (#1499, #1496) @liqikai9

Source code(tar.gz)
Source code(zip)
v1.0.0b0(Sep 1, 2022)
We are excited to announce the release of MMPose 1.0.0beta. MMPose 1.0.0beta is the first version of MMPose 1.x, a part of the OpenMMLab 2.0 projects. Built upon the new training engine.

Highlights

New engines. MMPose 1.x is based on MMEngine, which provides a general and powerful runner that allows more flexible customizations and significantly simplifies the entrypoints of high-level interfaces.

Unified interfaces. As a part of the OpenMMLab 2.0 projects, MMPose 1.x unifies and refactors the interfaces and internal logics of train, testing, datasets, models, evaluation, and visualization. All the OpenMMLab 2.0 projects share the same design in those interfaces and logics to allow the emergence of multi-task/modality algorithms.

More documentation and tutorials. We add a bunch of documentation and tutorials to help users get started more smoothly. Read it here.

Breaking Changes

In this release, we made lots of major refactoring and modifications. Please refer to the migration guide for details and migration instructions.
Source code(tar.gz)
Source code(zip)
v0.28.1(Jul 28, 2022)

This release is meant to fix the compatibility with the latest mmcv v1.6.1
Source code(tar.gz)
Source code(zip)
v0.28.0(Jul 6, 2022)
Highlights

Support TCFormer backbone, CVPR'2022 (#1447, #1452) @zengwang430521

Add RLE models on COCO dataset (#1424) @Indigo6, @Ben-Louis, @ly015

Update swin models with better performance (#1467) @jin-s13

New Features

Support TCFormer backbone, CVPR'2022 (#1447, #1452) @zengwang430521

Add RLE models on COCO dataset (#1424) @Indigo6, @Ben-Louis, @ly015

Support layer decay optimizer conctructor and learning rate decay optimizer constructor (#1423) @jin-s13

Improvements

Improve documentation quality (#1416, #1421, #1423, #1426, #1458, #1463) @ly015, @liqikai9

Support installation by mim (#1425) @liqikai9

Support PAVI logger (#1434) @EvelynWang-0423

Add progress bar for some demos (#1454) @liqikai9

Webcam API supports quick device setting in terminal commands (#1466) @ly015

Update swin models with better performance (#1467) @jin-s13

Bug Fixes

Rename custom_hooks_config to custom_hooks in configs to align with the documentation (#1427) @ly015

Fix deadlock issue in Webcam API (#1430) @ly015

Fix smoother configs in video 3D demo (#1457) @ly015

Source code(tar.gz)
Source code(zip)
v0.27.0(Jun 7, 2022)
Highlights

Support hand gesture recognition

Try the demo for gesture recognition

Learn more about the algorithm, dataset and experiment results

Major upgrade to the Webcam API

Tutorials (EN|zh_CN)

API Reference

Demo

New Features

Support gesture recognition algorithm MTUT CVPR'2019 and dataset NVGesture CVPR'2016 (#1380) @Ben-Louis

Improvements

Upgrade Webcam API and related documents (#1393, #1404, #1413) @ly015

Support exporting COCO inference result without the annotation file (#1368) @liqikai9

Replace markdownlint with mdformat in CI to avoid the dependence on ruby #1382 @ly015

Improve documentation quality (#1385, #1394, #1395, #1408) @chubei-oppen, @ly015, @liqikai9

Bug Fixes

Fix xywh->xyxy bbox conversion in dataset sanity check (#1367) @jin-s13

Fix a bug in two-stage 3D keypoint demo (#1373) @ly015

Fix out-dated settings in PVT configs (#1376) @ly015

Fix myst settings for document compiling (#1381) @ly015

Fix a bug in bbox transform (#1384) @ly015

Fix inaccurate description of min_keypoints in tracking apis (#1398) @pallgeuer

Fix warning with torch.meshgrid (#1402) @pallgeuer

Remove redundant transformer modules from mmpose.datasets.backbones.utils (#1405) @ly015

Source code(tar.gz)
Source code(zip)

v0.26.0(May 5, 2022)

Highlights

Support RLE (Residual Log-likelihood Estimation), ICCV'2021 (#1259) @Indigo6, @ly015
Support Swin Transformer, ICCV'2021 (#1300) @yumendecc, @ly015
Support PVT, ICCV'2021 and PVTv2, CVMJ'2022 (#1343) @zengwang430521
Speed up inference and reduce CPU usage by optimizing the pre-processing pipeline (#1320) @chenxinfeng4, @liqikai9

New Features

Support RLE (Residual Log-likelihood Estimation), ICCV'2021 (#1259) @Indigo6, @ly015
Support Swin Transformer, ICCV'2021 (#1300) @yumendecc, @ly015
Support PVT, ICCV'2021 and PVTv2, CVMJ'2022 (#1343) @zengwang430521
Support FPN, CVPR'2017 (#1300) @yumendecc, @ly015

Improvements

Speed up inference and reduce CPU usage by optimizing the pre-processing pipeline (#1320) @chenxinfeng4, @liqikai9
Video demo supports models that requires multi-frame inputs (#1300) @liqikai9, @jin-s13
Update benchmark regression list (#1328) @ly015, @liqikai9
Remove unnecessary warnings in TopDownPoseTrack18VideoDataset (#1335) @liqikai9
Improve documentation quality (#1313, #1305) @Ben-Louis, @ly015
Update deprecating settings in configs (#1317) @ly015

Bug Fixes

Fix a bug in human skeleton grouping that may skip the matching process unexpectedly when ignore_to_much is True (#1341) @daixinghome
Fix a GPG key error that leads to CI failure (#1354) @ly015
Fix bugs in distributed training script (#1338, #1298) @ly015
Fix an upstream bug in xtoccotools that causes incorrect AP(M) results (#1308) @jin-s13, @ly015
Fix indentiation errors in the colab tutorial (#1298) @YuanZi1501040205
Fix incompatible model weight initialization with other OpenMMLab codebases (#1329) @274869388
Fix HRNet FP16 checkpoints download URL (#1309) @YinAoXiong
Fix typos in body3d_two_stage_video_demo.py (#1295) @mucozcan

Breaking Changes

Refactor bbox processing in datasets and pipelines (#1311) @ly015, @Ben-Louis The bbox format conversion (xywh to center-scale) and random translation are moved from the dataset to the pipeline. The comparison between new and old version is as below:

	v0.26.0	v0.25.0
Dataset (e.g. TopDownCOCODataset)	`... # Data sample only contains bbox rec.append({ 'bbox': obj['clean_bbox][:4], ... })`	`... # Convert bbox from xywh to center-scale center, scale = self._xywh2cs(*obj['clean_bbox'][:4]) # Data sample contains center and scale rec.append({ 'bbox': obj['clean_bbox][:4], 'center': center, 'scale': scale, ... })`
Pipeline Config (e.g. HRNet+COCO)	`... train_pipeline = [ dict(type='LoadImageFromFile'), # Convert bbox from xywh to center-scale dict(type='TopDownGetBboxCenterScale', padding=1.25), # Randomly shift bbox center dict(type='TopDownRandomShiftBboxCenter', shift_factor=0.16, prob=0.3), ... ]`	`... train_pipeline = [ dict(type='LoadImageFromFile'), ... ]`
Advantage	Simpler data sample content Flexible bbox format conversion and augmentation Apply bbox random translation every epoch (instead of only applying once at the annotation loading)	-
BC Breaking	The method `_xywh2cs` of dataset base classes (e.g. Kpt2dSviewRgbImgTopDownDataset) will be deprecated in the future. Custom datasets will need modifications to move the bbox format conversion to pipelines.	-

Source code(tar.gz)
Source code(zip)

v0.25.1(Apr 22, 2022)

This release is meant to fix the compatibility with the latest mmcv v1.5.0
Source code(tar.gz)
Source code(zip)
v0.25.0(Apr 2, 2022)
Highlights

Support Shelf and Campus datasets with pre-trained VoxelPose models, "3D Pictorial Structures for Multiple Human Pose Estimation", CVPR'2014 (#1225) @liqikai9 @wusize

Add Smoother module for temporal smoothing of the pose estimation with configurable filters (#1127) @ailingzengzzz, @ly015

Support SmoothNet for pose smoothing, "SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos", arXiv'2021 (#1279) @ailingzengzzz, @ly015

Add multiview 3D pose estimation demo (#1270) @wusize

New Features

Support Shelf and Campus datasets with pre-trained VoxelPose models, "3D Pictorial Structures for Multiple Human Pose Estimation", CVPR'2014 (#1225) @liqikai9 @wusize

Add Smoother module for temporal smoothing of the pose estimation with configurable filters (#1127) @ailingzengzzz, @ly015

Support SmoothNet for pose smoothing, "SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos", arXiv'2021 (#1279) @ailingzengzzz, @ly015

Add multiview 3D pose estimation demo (#1270) @wusize

Support multi-machine distributed training (#1248) @ly015

Improvements

Update HRFormer configs and checkpoints with relative position bias (#1245) @zengwang430521

Support using different random seed for each distributed node (#1257, #1229) @ly015

Improve documentation quality (#1275, #1255, #1258, #1249, #1247, #1240, #1235) @ly015, @jin-s13, @YoniChechik

Bug Fixes

Fix keypoint index in RHD dataset meta information (#1265) @liqikai9

Fix pre-commit hook unexpected behavior on Windows (#1282) @liqikai9

Remove python-dev installation in CI (#1276) @ly015

Unify hyphens in argument names in tools and demos (#1271) @ly015

Fix ambiguous channel size in channel_shuffle that may cause exporting failure (#1242) @PINTO0309

Fix a bug in Webcam API that causes single-class detectors fail (#1239) @674106399

Fix the issue that custom_hook can not be set in configs (#1236) @bladrome

Fix incompatible MMCV version in DockerFile (#raykindle)

Skip invisible joints in visualization (#1228) @womeier

Source code(tar.gz)
Source code(zip)
v0.24.0(Mar 7, 2022)
Highlights

Support HRFormer "HRFormer: High-Resolution Vision Transformer for Dense Predict", NeurIPS'2021 (#1203) @zengwang430521

Support Windows installation with pip (#1213) @jin-s13, @ly015

Add WebcamAPI documents and tutorials (#1187) @ly015

New Features

Support HRFormer "HRFormer: High-Resolution Vision Transformer for Dense Predict", NeurIPS'2021 (#1203) @zengwang430521

Support Windows installation with pip (#1213) @jin-s13, @ly015

Support CPU training with mmcv < v1.4.4 (#1161) @EasonQYS, @ly015

Add "Valentine Magic" demo with WebcamAPI (#1189, #1191) @liqikai9

Improvements

Refactor multi-view 3D pose estimation framework towards better modularization and expansibility (#1196) @wusize

Add WebcamAPI documents and tutorials (#1187) @ly015

Refactor dataset evaluation interface to align with other OpenMMLab codebases (#1209) @ly015

Add deprecation message for deploy tools since MMDeploy has supported MMPose (#1207) @QwQ2000

Improve documentation quality (#1206, #1161) @ly015

Switch to OpenMMLab official pre-commit-hook for copyright check (#1214) @ly015

Bug Fixes

Fix hard-coded data collating and scattering in inference (#1175) @ly015

Fix model configs on JHMDB dataset (#1188) @jin-s13

Fix area calculation in pose tracking inference (#1197) @pallgeuer

Fix registry scope conflict of module wrapper (#1204) @ly015

Update MMCV installation in CI and documents (#1205)

Fix incorrect color channel order in visualization functions (#1212) @ly015

Source code(tar.gz)
Source code(zip)
v0.23.0(Feb 14, 2022)
Highlights

Add MMPose Webcam API: A simple yet powerful tools to develop interactive webcam applications with MMPose functions. (#1178, #1173, #1173, #1143, #1094, #1133, #1098, #1160) @ly015, @jin-s13, @liqikai9, @wusize, @luminxu, @zengwang430521 @mzr1996

New Features

Add MMPose Webcam API: A simple yet powerful tools to develop interactive webcam applications with MMPose functions. (#1178, #1173, #1173, #1143, #1094, #1133, #1098, #1160) @ly015, @jin-s13, @liqikai9, @wusize, @luminxu, @zengwang430521 @mzr1996

Support ConcatDataset (#1139) @Canwang-sjtu

Support CPU training and testing (#1157) @ly015

Improvements

Add multi-processing configurations to speed up distributed training and testing (#1146) @ly015

Add default runtime config (#1145)

Upgrade isort in pre-commit hook (#1179) @liqikai9

Update README and documents (#1171, #1167, #1153, #1149, #1148, #1147, #1140) @jin-s13, @wusize, @TommyZihao, @ly015

Bug Fixes

Fix undeterministic behavior in pre-commit hooks (#1136) @jin-s13

Deprecate the support for "python setup.py test" (#1179) @ly015

Fix incompatible settings with MMCV on HSigmoid default parameters (#1132) @ly015

Fix albumentation installation (#1184) @BIGWangYuDong

Source code(tar.gz)
Source code(zip)
v0.22.0(Jan 5, 2022)
Highlights

Support VoxelPose "VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment", ECCV'2020 (#1050) @wusize

Support Soft Wing loss "Structure-Coherent Deep Feature Learning for Robust Face Alignment", TIP'2021 (#1077) @jin-s13

Support Adaptive Wing loss "Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression", ICCV'2019 (#1072) @jin-s13

New Features

Support VoxelPose "VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment", ECCV'2020 (#1050) @wusize

Support Soft Wing loss "Structure-Coherent Deep Feature Learning for Robust Face Alignment", TIP'2021 (#1077) @jin-s13

Support Adaptive Wing loss "Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression", ICCV'2019 (#1072) @jin-s13

Add LiteHRNet-18 Checkpoints trained on COCO. (#1120) @jin-s13

Improvements

Improve documentation quality (#1115, #1111, #1105, #1087, #1086, #1085, #1084, #1083, #1124, #1070, #1068) @jin-s13, @liqikai9, @ly015

Support CircleCI (#1074) @ly015

Skip unit tests in CI when only document files were changed (#1074, #1041) @QwQ2000, @ly015

Support file_client_args in LoadImageFromFile (#1076) @jin-s13

Bug Fixes

Fix a bug in Dark UDP postprocessing that causes error when the channel number is large. (#1079, #1116) @X00123, @jin-s13

Fix hard-coded sigmas in bottom-up image demo (#1107, #1101) @chenxinfeng4, @liqikai9

Fix unstable checks in unit tests (#1112) @ly015

Do not destroy NULL windows if args.show==False in demo scripts (#1104) @bladrome

Source code(tar.gz)
Source code(zip)
v0.21.0(Dec 6, 2021)
Highlights

Support "Learning Temporal Pose Estimation from Sparsely-Labeled Videos", NeurIPS'2019 (#932, #1006, #1036, #1060) @liqikai9

Add ViPNAS-MobileNetV3 models (#1025) @luminxu, @jin-s13

Add inference speed benchmark (#1028, #1034, #1044) @liqikai9

New Features

Support "Learning Temporal Pose Estimation from Sparsely-Labeled Videos", NeurIPS'2019 (#932, #1006, #1036) @liqikai9

Add ViPNAS-MobileNetV3 models (#1025) @luminxu, @jin-s13

Add light-weight top-down models for whole-body keypoint detection (#1009, #1020, #1055) @luminxu, @ly015

Add HRNet checkpoints with various settings on PoseTrack18 (#1035) @liqikai9

Improvements

Add inference speed benchmark (#1028, #1034, #1044) @liqikai9

Update model metafile format (#1001) @ly015

Support minus output feature index in mobilenet_v3 (#1005) @luminxu

Improve documentation quality (#1018, #1026, #1027, #1031, #1038, #1046, #1056, #1057) @edybk, @luminxu, @ly015, @jin-s13

Set default random seed in training initialization (#1030) @ly015

Skip CI when only specific files changed (#1041, #1059) @QwQ2000, @ly015

Automatically cancel uncompleted action runs when new commit arrives (#1053) @ly015

Bug Fixes

Update pose tracking demo to be compatible with latest mmtracking (#1014) @jin-s13

Fix symlink creation failure when installed in Windows environments (#1039) @QwQ2000

Fix AP-10K dataset sigmas (#1040) @jin-s13

Source code(tar.gz)
Source code(zip)
v0.20.0(Nov 1, 2021)
Highlights

Add AP-10K dataset for animal pose estimation (#987) @Annbless, @AlexTheBad, @jin-s13, @ly015

Support TorchServe (#979) @ly015

New Features

Add AP-10K dataset for animal pose estimation (#987) @Annbless, @AlexTheBad, @jin-s13, @ly015

Add HRNetv2 checkpoints on 300W and COFW datasets (#980) @jin-s13

Support TorchServe (#979) @ly015

Bug Fixes

Fix some deprecated or risky settings in configs (#963, #976, #992) @jin-s13, @wusize

Fix issues of default arguments of training and testing scripts (#970, #985) @liqikai9, @wusize

Fix heatmap and tag size mismatch in bottom-up with UDP (#994) @wusize

Fix python3.9 installation in CI (#983) @ly015

Fix model zoo document integrity issue (#990) @jin-s13

Improvements

Support non-square input shape for bottom-up (#991) @wusize

Add image and video resources for demo (#971) @liqikai9

Use CUDA docker images to accelerate CI (#973) @ly015

Add codespell hook and fix detected typos (#977) @ly015

Source code(tar.gz)
Source code(zip)
v0.19.0(Oct 8, 2021)
Highlights

Add models for Associative Embedding with Hourglass network backbone (#906, #955) @jin-s13, @luminxu

Support COCO-Wholebody-Face and COCO-Wholebody-Hand datasets (#813) @jin-s13, @innerlee, @luminxu

Upgrade dataset interface (#901, #924) @jin-s13, @innerlee, @ly015, @liqikai9

New style of documentation (#945) @ly015

New Features

Add models for Associative Embedding with Hourglass network backbone (#906, #955) @jin-s13, @luminxu

Support COCO-Wholebody-Face and COCO-Wholebody-Hand datasets (#813) @jin-s13, @innerlee, @luminxu

Add pseudo-labeling tool to generate COCO style keypoint annotations with given bounding boxes (#928) @soltkreig

New style of documentation (#945) @ly015

Bug Fixes

Fix segmentation parsing in Macaque dataset preprocessing (#948) @jin-s13

Fix dependencies that may lead to CI failure in downstream projects (#936, #953) @RangiLyu, @ly015

Fix keypoint order in Human3.6M dataset (#940) @ttxskk

Fix unstable image loading for Interhand2.6M (#913) @zengwang430521

Improvements

Upgrade dataset interface (#901, #924) @jin-s13, @innerlee, @ly015, @liqikai9

Improve demo usability and stability (#908, #934) @ly015

Standardize model metafile format (#941) @ly015

Support persistent_worker and several other arguments in configs (#946) @jin-s13

Use MMCV root model registry to enable cross-project module building (#935) @RangiLyu

Improve the document quality (#916, #909, #942, #913, #956) @jin-s13, @ly015, @bit-scientist, @zengwang430521

Improve pull request template (#952, #954) @ly015

Breaking Changes

Upgrade dataset interface (#901) @jin-s13, @innerlee, @ly015

Source code(tar.gz)
Source code(zip)
v0.18.0(Sep 2, 2021)
Improvements

Add automatic benchmark regression tools (#849, #880, #885) @liqikai9, @ly015

Add copyright information and checking hook (#872)

Add PR template (#875) @ly015

Add citation information (#876) @ly015

Add python3.9 in CI (#877, #883) @ly015

Improve the quality of the documents (#845, #845, #848, #867, #870, #873, #896) @jin-s13, @ly015, @zhiqwang

Bug Fixes

Fix redundant model weight loading in pytorch-to-onnx conversion (#850) @ly015

Fix a bug in update_model_index.py that may cause pre-commit hook failure(#866) @ly015

Fix a bug in interhand_3d_head (#890) @zengwang430521

Fix pose tracking demo failure caused by out-of-date configs (#891)

Source code(tar.gz)
Source code(zip)
v0.17.0(Aug 6, 2021)
Highlights

Support "Lite-HRNet: A Lightweight High-Resolution Network" CVPR'2021 (#733,#800) @jin-s13

Add 3d body mesh demo (#771) @zengwang430521

Add Chinese documentation (#787, #798, #799, #802, #804, #805, #815, #816, #817, #819, #839) @ly015, @luminxu, @jin-s13, @liqikai9, @zengwang430521

Add Colab Tutorial (#834) @ly015

New Features

Support "Lite-HRNet: A Lightweight High-Resolution Network" CVPR'2021 (#733,#800) @jin-s13

Add 3d body mesh demo (#771) @zengwang430521

Add Chinese documentation (#787, #798, #799, #802, #804, #805, #815, #816, #817, #819, #839) @ly015, @luminxu, @jin-s13, @liqikai9, @zengwang430521

Add Colab Tutorial (#834) @ly015

Support training for InterHand v1.0 dataset (#761) @zengwang430521

Bug Fixes

Fix mpii [email protected] index (#773) @jin-s13

Fix multi-node distributed test (#818) @ly015

Fix docstring and init_weights error of ShuffleNetV1 (#814) @Junjun2016

Fix imshow_bbox error when input bboxes is empty (#796) @ly015

Fix model zoo doc generation (#778) @ly015

Fix typo (#767), (#780, #782) @ly015, @jin-s13

Breaking Changes

Use MMCV EvalHook (#686) @ly015

Improvements

Add pytest.ini and fix docstring (#812) @jin-s13

Support None as target weights in losses (#829) @Ezra-Yu

Move process_mmdet_results into inference.py (#831) @ly015

Update resource limit (#783) @jin-s13

Use COCO 2D pose model in 3D demo examples (#785) @ly015

Change model zoo titles in the doc from center-aligned to left-aligned (#792, #797) @ly015

Support MIM (#706, #794) @ly015

Update out-of-date configs (#827) @jin-s13

Remove opencv-python-headless dependency by albumentations (#833) @ly015

Update QQ QR code in README_CN.md (#832) @ly015

Source code(tar.gz)
Source code(zip)
v0.16.0(Jul 3, 2021)
Demo Video

Highlights

Support "ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search" CVPR'2021 (#742,#755).

Support MPI-INF-3DHP dataset (#683,#746,#751).

Add webcam demo tool (#729)

Add 3d body and hand pose estimation demo (#704, #727).

New Features

Support "ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search" CVPR'2021 (#742,#755)

Support MPI-INF-3DHP dataset (#683,#746,#751)

Support Webcam demo (#729)

Support Interhand 3d demo (#704)

Support 3d pose video demo (#727)

Support H36m dataset for 2d pose estimation (#709,#735)

Add scripts to generate mim metafile (#749)

Bug Fixes

Fix typos (#692,#696,#697,#698,#712,#718,#728)

Change model download links from http to https (#716)

Breaking Changes

Switch to MMCV MODEL_REGISTRY (#669)

Improvements

Refactor MeshMixDataset (#752)

Rename 'GaussianHeatMap' to 'GaussianHeatmap' (#745)

Update out-of-date configs (#734)

Improve compatibility for breaking changes (#731)

Enable to control radius and thickness in visualization (#722)

Add regex dependency (#720)

Source code(tar.gz)
Source code(zip)
0.15.0(Jun 3, 2021)
Highlights

Support 3d video pose estimation (VideoPose3D).

Support 3d hand pose estimation (InterNet).

Improve presentation of the model zoo.

New Features

Support "InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image" (ECCV‘20) (#624)

Support "3D human pose estimation in video with temporal convolutions and semi-supervised training" (CVPR'19) (#602,#681)

Support 3d pose estimation demo (#653,#670)

Support bottom-up whole-body pose estimation (#689)

Support mmcli (#634)

Bug Fixes

Fix opencv compatibility (#635)

Fix demo with UDP (#637)

Fix bottom-up model onnx convertion (#680)

Fix GPU_IDS in distributed training (#668)

Fix MANIFEST.in (#641,#657)

Fix docs (#643,#684,#688,#690,#692)

Breaking Changes

Reorganize configs by tasks, algorithms, datasets, and techniques (#647)

Rename heads and detectors (#667)

Improvements

Add radius and thickness parameters in visualization (#638)

Add trans_prob parameter in TopDownRandomTranslation (#650)

Switch to MMCV MODEL_REGISTRY (#669)

Update dependencies (#674,#676)

Source code(tar.gz)
Source code(zip)
v0.14.0(May 7, 2021)
Highlights

Support animal pose estimation with 7 popular datasets.

Support "A simple yet effective baseline for 3d human pose estimation" (ICCV'17).

New Features

Support "A simple yet effective baseline for 3d human pose estimation" (ICCV'17) (#554,#558,#566,#570,#589)

Support animal pose estimation (#559,#561,#563,#571,#603,#605)

Support Horse-10 dataset (#561), MacaquePose dataset (#561), Vinegar Fly dataset (#561), Desert Locust dataset (#561), Grevy's Zebra dataset (#561), ATRW dataset (#571), and Animal-Pose dataset (#603).

Support bottom-up pose tracking demo (#574).

Support FP16 training (#584,#616,#626).

Support NMS for bottom-up (#609).

Bug Fixes

Fix bugs in the top-down demo, when there are no people in the images (#569).

Fix the links in the doc (#612).

Improvements

Speed up top-down inference (#560)

Update github CI (#562,#564)

Update Readme (#578,#579,#580,#592,#599,#600,#607)

Update Faq (#587,#610)

Source code(tar.gz)
Source code(zip)
v0.13.0(Apr 1, 2021)
Highlights

Support Wingloss.

Support RHD hand dataset.

New Features

Support Wingloss (#482)

Support RHD hand dataset (#523, #551)

Support Human3.6m dataset for 3d keypoint detection (#518, #527)

Support TCN model for 3d keypoint detection (#521, #522)

Support Interhand3D model for 3d hand detection (#536)

Support Multi-task detector (#480)

Bug Fixes

Fix [email protected] calculation (#516)

Fix unittest (#529)

Fix circular importing (#542)

Fix bugs in bottom-up keypoint score (#548)

Improvements

Update config & checkpoints (#525, #546)

Fix typos (#514, #519, #532, #537, )

Speed up post processing (#535)

Update mmcv version dependency (#544)

Source code(tar.gz)
Source code(zip)
v0.12.0(Mar 1, 2021)
Highlights

Support DeepPose algorithm.

New Features

Support DeepPose algorithm (#446, #461)

Support interhand3d dataset (#468)

Support Albumentation pipeline (#469)

Support PhotometricDistortion pipeline (#485)

Set seed option for training (#493)

Add demos for face keypoint detection (#502)

Bug Fixes

Change channel order according to configs (#504)

Fix num_factors in UDP encoding (#495)

Fix configs (#456)

Breaking Changes

Refactor configs for wholebody pose estimation (#487, #491)

Rename decode function for heads (#481)

Improvements

Update config & checkpoints (#453,#484,#487)

Add README in Chinese (#462)

Add tutorials about configs (#465)

Add demo videos for various tasks (#499, #503)

Update docs about MMPose installation (#467, #505)

Rename stat.py to stats.py (#483)

Fix typos (#463, #464, #477, #481)

latex to bibtex (#471)

Update FAQ (#466)

Source code(tar.gz)
Source code(zip)
v0.11.0(Feb 1, 2021)
Highlights

Support fashion landmark detection.

Support face keypoint detection.

Support pose tracking with MMTracking.

New Features

Support fashion landmark detection (DeepFashion) (#413)

Support face keypoint detection (300W, AFLW, COFW, WFLW) (#367)

Support pose tracking demo with MMTracking (#427)

Support face demo (#443)

Support AIC dataset for bottom-up methods (#438, #449)

Bug Fixes

Fix multi-batch training (#434)

Fix sigmas in AIC dataset (#441)

Fix config file (#420)

Breaking Changes

Refactor Heads (#382)

Improvements

Update documentation (#409, #412, #415, #416, #419, #421, #422, #424, #425, #435, #436, #437, #444, #445)

Add GAP (global average pooling) neck (#414)

Speed up (#411, #423)

Support COCO test-dev test (#433)

Source code(tar.gz)
Source code(zip)
v0.10.0(Jan 5, 2021)
Highlights

Support more human pose estimation methods.

UDP

Support pose tracking.

Support multi-batch inference.

Add some useful tools, including analyze_logs, get_flops, print_config.

Support more backbone networks.

ResNest

VGG

New Features

Support UDP (#353, #371, #402)

Support multi-batch inference (#390)

Support MHP dataset (#386)

Support pose tracking demo (#380)

Support MPII-TRB demo (#372)

Support MobileNet for hand pose estimation (#377)

Support ResNest backbone (#370)

Support VGG backbone (#370)

Add some useful tools, including analyze_logs, get_flops, print_config (#324)

Bug Fixes

Fix bugs in pck evaluation (#328)

Fix model download links in README (#396, #397)

Fix CrowdPose annotations and update benchmarks (#384)

Fix modelzoo stat (#354, #360, #362)

Fix config files for AIC datasets (#340)

Breaking Changes

Rename image_thr to det_bbox_thr for top-down methods.

Improvements

Organize the readme files (#398, #399, #400)

Check linting for markdown (#379)

Add faq.md (#350)

Remove PyTorch 1.4 in CI (#338)

Add pypi badge in readme (#329)

Source code(tar.gz)
Source code(zip)
v0.9.0(Nov 30, 2020)
Highlights

Support more human pose estimation methods.

MSPN

RSN

Support video pose estimation datasets.

sub-JHMDB

Support Onnx model conversion.

New Features

Support MSPN (#278)

Support RSN (#221, #318)

Support new post-processing method for MSPN & RSN (#288)

Support sub-JHMDB dataset (#292)

Support urls for pre-trained models in config files (#232)

Support Onnx (#305)

Bug Fixes

Fix model download links in README (#255, #315)

Breaking Changes

post_process=True|False and unbiased_decoding=True|False are deprecated, use post_process=None|default|unbiased etc. instead (#288)

Improvements

Enrich the model zoo (#256, #320)

Set the default map_location as 'cpu' to reduce gpu memory cost (#227)

Support return heatmaps and backbone features for bottom-up models (#229)

Upgrade mmcv maximum & minimum version (#269, #313)

Automatically add modelzoo statistics to readthedocs (#252)

Fix Pylint issues (#258, #259, #260, #262, #265, #267, #268, #270, #271, #272, #273, #275, #276, #283, #285, #293, #294, #295)

Improve README (#226, #257, #264, #280, #296)

Support PyTorch 1.7 in CI (#274)

Add docs/tutorials for running demos (#263)

Source code(tar.gz)
Source code(zip)
v0.8.0(Oct 31, 2020)
Highlights

Support more human pose estimation datasets.

CrowdPose

PoseTrack18

Support more 2D hand keypoint estimation datasets.

InterHand2.6

Support adversarial training for 3D human shape recovery.

Support multi-stage losses.

Support mpii demo.

New Features

Support CrowdPose dataset (#195).

Support PoseTrack18 dataset (#220).

Support InterHand2.6 dataset (#202).

Support adversarial training for 3D human shape recovery (#192).

Support multi-stage losses (#204).

Bug Fixes

Fix config files (#190)

Improvements

Add mpii demo (#216)

Improve README (#181, #183, #208)

Support return heatmaps and backbone features (#196, #212)

Support different return formats of mmdetection models (#217)

Source code(tar.gz)
Source code(zip)
v0.7.0(Oct 5, 2020)
Highlights

Support HMR for 3D human shape recovery.

Support WholeBody human pose estimation.

COCO-WholeBody

Support more 2D hand keypoint estimation datasets.

Frei-hand

CMU Panoptic HandDB

Add more popular backbones & enrich the modelzoo.

ShuffleNetv2

Support hand demo and whole-body demo.

New Features

Support HMR for 3D human shape recovery (#157, #160, #161, #162)

Support COCO-WholeBody dataset (#133)

Support Frei-hand dataset (#125)

Support CMU Panoptic HandDB dataset (#144)

Support H36M dataset (#159)

Support ShuffleNetv2 (#139)

Support saving best models based on key indicator (#127)

Bug Fixes

Fix typos in docs (#121)

Fix assertion (#142)

Improvements

Add tools to transform .mat format to .json format (#126)

Add hand demo (#115)

Add whole-body demo (#163)

Reuse mmcv utility function and update version files (#135, #137)

Enrich the modelzoo (#147, #169)

Improve docs (#174, #175, #178)

Improve README (#176)

Improve version.py (#173)

Source code(tar.gz)
Source code(zip)
v0.6.0(Sep 2, 2020)
Highlights

Add popular backbones & enrich the modelzoo.

ResNext

SEResNet

ResNetV1D

MobileNetv2

ShuffleNetv1

CPM (Convolutional Pose Machine)

Add popular datasets:

AIChallenger

MPII

MPII-TRB

OCHuman

Support 2d hand keypoint estimation.

OneHand10K

Support bottom-up inference.

New Features

Support OneHand10K dataset (#52).

Support MPII dataset (#55).

Support MPII-TRB dataset (#19, #47, #48).

Support OCHuman dataset (#70).

Support AIChallenger dataset (#87).

Support multiple backbones (#26).

Support CPM model (#56).

Bug Fixes

Fix configs for MPII & MPII-TRB datasets (#93).

Fix the bug of missing test_pipeline in configs (#14).

Fix typos (#27, #28, #50, #53, #63).

Improvements

Update benchmark (#93).

Add Dockerfile (#44).

Improve unittest coverage and minor fix (#18).

Support CPUs for train/val/demo (#34).

Support bottom-up demo (#69).

Add tools to publish model (#62).

Enrich the modelzoo (#64, #68, #82).

Source code(tar.gz)
Source code(zip)

OpenMMLab Pose Estimation Toolbox and Benchmark.

Related tags

Overview

Introduction

Major Features

Benchmark

Installation

Data Preparation

Get Started

FAQ

License

Citation

Contributing

Acknowledgement

Projects in OpenMMLab

Comments

Motivation

Modification

Result

Limitation

Checklist

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Releases(v1.0.0rc0)

v1.0.0rc0(Oct 14, 2022)

v0.29.0(Oct 14, 2022)

v1.0.0b0(Sep 1, 2022)

v0.28.1(Jul 28, 2022)

v0.28.0(Jul 6, 2022)

v0.27.0(Jun 7, 2022)

v0.26.0(May 5, 2022)

v0.25.1(Apr 22, 2022)

v0.25.0(Apr 2, 2022)

v0.24.0(Mar 7, 2022)

v0.23.0(Feb 14, 2022)

v0.22.0(Jan 5, 2022)

v0.21.0(Dec 6, 2021)

v0.20.0(Nov 1, 2021)

v0.19.0(Oct 8, 2021)

v0.18.0(Sep 2, 2021)

v0.17.0(Aug 6, 2021)

v0.16.0(Jul 3, 2021)

0.15.0(Jun 3, 2021)

v0.14.0(May 7, 2021)

v0.13.0(Apr 1, 2021)

v0.12.0(Mar 1, 2021)

v0.11.0(Feb 1, 2021)

v0.10.0(Jan 5, 2021)

v0.9.0(Nov 30, 2020)

v0.8.0(Oct 31, 2020)

v0.7.0(Oct 5, 2020)

Highlights

New Features

Bug Fixes

Improvements

v0.6.0(Sep 2, 2020)

Owner

OpenMMLab

Brain tumor detection using CNN (InceptionResNetV2 Model)

CryptoFrog - My First Strategy for freqtrade

Neural Magic Eye: Learning to See and Understand the Scene Behind an Autostereogram, arXiv:2012.15692.

A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019).

Local trajectory planner based on a multilayer graph framework for autonomous race vehicles.

The code for paper "Contrastive Spatio-Temporal Pretext Learning for Self-supervised Video Representation" which is accepted by AAAI 2022

FAVD: Featherweight Assisted Vulnerability Discovery

Official PyTorch Implementation of paper "NeLF: Neural Light-transport Field for Single Portrait View Synthesis and Relighting", EGSR 2021.

A curated (most recent) list of resources for Learning with Noisy Labels

CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images

This is the official implement of paper "ActionCLIP: A New Paradigm for Action Recognition"

The official re-implementation of the Neurips 2021 paper, "Targeted Neural Dynamical Modeling".

Official implementation of "An Image is Worth 16x16 Words, What is a Video Worth?" (2021 paper)

PyTorch original implementation of Cross-lingual Language Model Pretraining.

Pacman-AI - AI project designed by UC Berkeley. Designed reflex and minimax agents for the game Pacman.

Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.

[BMVC2021] The official implementation of "DomainMix: Learning Generalizable Person Re-Identification Without Human Annotations"

This is implementation of AlexNet(2012) with 3D Convolution on TensorFlow (AlexNet 3D).

Solutions of Reinforcement Learning 2nd Edition