(ICCV 2021) Official code of "Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing."

Overview

Dressing in Order (DiOr)

👚 [Paper] 👖 [Webpage] 👗 [Running this code]

The official implementation of "Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing". by Aiyu Cui, Daniel McKee and Svetlana Lazebnik. (ICCV 2021)

🔔 Updates

Supported Try-on Applications

Supported Editing Applications

More results

Play with demo.ipynb!


Get Started

Please follow the installation instruction in GFLA to install the environment.

Then run

pip install -r requirements.txt

If one wants to run inference only: You can use later version of PyTorch and you don't need to worry about how to install GFLA's cuda functions. Please specify --frozen_flownet.

Dataset

We run experiments on Deepfashion Dataset. To set up the dataset:

  1. Download and unzip img_highres.zip from the deepfashion inshop dataset at $DATA_ROOT
  2. Download the train/val split and pre-processed keypoints annotations from GFLA source or PATN source, and put the .csv and .lst files at $DATA_ROOT.
    • If one wants to extract the keypoints from scratch, please run OpenPose as the pose estimator. Please follow the instruction from PATN for how to generate the keypoints in desired format.
  3. Run python tools/generate_fashion_dataset.py to split the data. (Please specify the $DATA_ROOT accordingly.)
  4. Get human parsing. You can obtain the parsing by either:
    • Run off-the-shelf human parser SCHP (with LIP labels) on $DATA_ROOT/train and $DATA_ROOT/test. Name the output parses folder as $DATA_ROOT/trainM_lip and $DATA_ROOT/testM_lip respectively.
    • Download the preprocessed parsing from here and put it under $DATA_ROOT.
  5. Download standard_test_anns.txt for fast visualization.

After the processing, you should have the dataset folder formatted like:

+ $DATA_ROOT
|   + train (all training images)
|   |   - xxx.jpg
|   |     ...
|   + trainM_lip (human parse of all training images)
|   |   - xxx.png
|   |     ...
|   + test (all test images)
|   |   - xxx.jpg
|   |     ...
|   + testM_lip (human parse of all test images)
|   |   - xxx.png
|   |     ...
|   - fashion-pairs-train.csv (paired poses for training)
|   - fashion-pairs-test.csv (paired poses for test)
|   - fashion-annotation-train.csv (keypoints for training images)
|   - fashion-annotation-test.csv  (keypoints for test images)
|   - train.lst
|   - test.lst
|   - standard_test_anns.txt

Run Demo

Please download the pretrained weights from here and unzip at checkpoints/.

After downloading the pretrained model and setting the data, you can try out our applications in notebook demo.ipynb.

(The checkpoints above are reproduced, so there could be slightly difference in quantitative evaluation from the reported results. To get the original results, please check our released generated images here.)

(DIORv1_64 was trained with a minor difference in code, but it may give better visual results in some applications. If one wants to try it, specify --netG diorv1.)


Training

Warmup the Global Flow Field Estimator

Note, if you don't want to warmup the Global Flow Field Estimator, you can extract its weights from GFLA by downloading the pretrained weights GFLA from here.

Otherwise, run

sh scripts/run_pose.sh

Training

After warming up the flownet, train the pipeline by

sh scripts/run_train.sh

Run tensorboard --logdir checkpoints/$EXP_NAME/train to check tensorboard. Resetting discriminators may help training when it stucks at local minimals.

Evaluations

To download our generated images (256x176 reported in paper): here.

SSIM, FID and LPIPS

To run evaluation (SSIM, FID and LPIPS) on pose transfer task:

sh scripts/run_eval.sh

Cite us!

If you find this work is helpful, please consider to star 🌟 this repo and cite us as

@article{cui2021dressing,
  title={Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing},
  author={Cui, Aiyu and McKee, Daniel and Lazebnik, Svetlana},
  journal={arXiv preprint arXiv:2104.07021},
  year={2021}
}

Acknowledgements

This repository is built up on GFLA, pytorch-CycleGAN-and-pix2pix, PATN and MUNIT. Please be aware of their licenses when using the code.

Thanks a lot for the great work to the pioneer researchers!

Pixel Consensus Voting for Panoptic Segmentation (CVPR 2020)

Implementation for Pixel Consensus Voting (CVPR 2020). This codebase contains the essential ingredients of PCV, including various spatial discretizati

Haochen 23 Oct 25, 2022
A collection of Jupyter notebooks to play with NVIDIA's StyleGAN3 and OpenAI's CLIP for a text-based guided image generation.

StyleGAN3 CLIP-based guidance StyleGAN3 + CLIP StyleGAN3 + inversion + CLIP This repo is a collection of Jupyter notebooks made to easily play with St

Eugenio Herrera 176 Dec 30, 2022
Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Lane Follower This code is for the lane follower, including perception and control, as shown below. Environment Hardware Industrial Camera Intel-NUC(1

Siqi Fan 3 Jul 07, 2022
This is a collection of all challenges in HKCERT CTF 2021

香港網絡保安新生代奪旗挑戰賽 2021 (HKCERT CTF 2021) This is a collection of all challenges (and writeups) in HKCERT CTF 2021 Challenges ID Chinese name Name Score S

10 Jan 27, 2022
DecoupledNet is semantic segmentation system which using heterogeneous annotations

DecoupledNet: Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation Created by Seunghoon Hong, Hyeonwoo Noh and Bohyung Han at POSTE

Hyeonwoo Noh 74 Sep 22, 2021
Dense Gaussian Processes for Few-Shot Segmentation

DGPNet - Dense Gaussian Processes for Few-Shot Segmentation Welcome to the public repository for DGPNet. The paper is available at arxiv: https://arxi

37 Jan 07, 2023
Example how to deploy deep learning model with aiohttp.

aiohttp-demos Demos for aiohttp project. Contents Imagetagger Deep Learning Image Classifier URL shortener Toxic Comments Classifier Moderator Slack B

aio-libs 661 Jan 04, 2023
Do Neural Networks for Segmentation Understand Insideness?

This is part of the code to reproduce the results of the paper Do Neural Networks for Segmentation Understand Insideness? [pdf] by K. Villalobos (*),

biolins 0 Mar 20, 2021
[CVPR 2020] GAN Compression: Efficient Architectures for Interactive Conditional GANs

GAN Compression project | paper | videos | slides [NEW!] GAN Compression is accepted by T-PAMI! We released our T-PAMI version in the arXiv v4! [NEW!]

MIT HAN Lab 1k Jan 07, 2023
TensorFlow implementation of AlexNet and its training and testing on ImageNet ILSVRC 2012 dataset

AlexNet training on ImageNet LSVRC 2012 This repository contains an implementation of AlexNet convolutional neural network and its training and testin

Matteo Dunnhofer 161 Nov 25, 2022
Background-Click Supervision for Temporal Action Localization

Background-Click Supervision for Temporal Action Localization This repository is the official implementation of BackTAL. In this work, we study the te

LeYang 221 Oct 09, 2022
Customer-Transaction-Analysis - This analysis is based on a synthesised transaction dataset containing 3 months worth of transactions for 100 hypothetical customers.

Customer-Transaction-Analysis - This analysis is based on a synthesised transaction dataset containing 3 months worth of transactions for 100 hypothetical customers. It contains purchases, recurring

Ayodeji Yekeen 1 Jan 01, 2022
Woosung Choi 63 Nov 14, 2022
OptaPlanner wrappers for Python. Currently significantly slower than OptaPlanner in Java or Kotlin.

OptaPy is an AI constraint solver for Python to optimize the Vehicle Routing Problem, Employee Rostering, Maintenance Scheduling, Task Assignment, School Timetabling, Cloud Optimization, Conference S

OptaPy 211 Jan 02, 2023
These are the materials for the paper "Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations"

Few-shot-NLEs These are the materials for the paper "Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations". You can find the smal

Yordan Yordanov 0 Oct 21, 2022
Deep Multimodal Neural Architecture Search

MMNas: Deep Multimodal Neural Architecture Search This repository corresponds to the PyTorch implementation of the MMnas for visual question answering

Vision and Language Group@ MIL 23 Dec 21, 2022
Face Recognition plus identification simply and fast | Python

PyFaceDetection Face Recognition plus identification simply and fast Ubuntu Setup sudo pip3 install numpy sudo pip3 install cmake sudo pip3 install dl

Peyman Majidi Moein 16 Sep 22, 2022
Mscp jamf - Build compliance in jamf

mscp_jamf Build compliance in Jamf. This will build the following xml pieces to

Bob Gendler 3 Jul 25, 2022
Neon: an add-on for Lightbulb making it easier to handle component interactions

Neon Neon is an add-on for Lightbulb making it easier to handle component interactions. Installation pip install git+https://github.com/neonjonn/light

Neon Jonn 9 Apr 29, 2022
Official implementation of EfficientPose

EfficientPose This is the official implementation of EfficientPose. We based our work on the Keras EfficientDet implementation xuannianz/EfficientDet

2 May 17, 2022