✂️ EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video.

Overview

EyeLipCropper

EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video. The whole process consists of three parts: frame extraction, face alignment, and eye/mouth cropping. The cropped eye/mouth image size can be customized.

vis

Usage

Prerequisites

>>> pip install -r requirements.txt

1. Extract frames of a given video

>>> python frame_extract.py -h
usage: frame_extract.py [-h] [--video-path VIDEO_PATH] [--images-path IMAGES_PATH]

extract frames with opencv

optional arguments:
  -h, --help            show this help message and exit
  --video-path VIDEO_PATH
                        the input video path
  --images-path IMAGES_PATH
                        the output frames path
 
# default for test: this will generate frames of the video in `./test/images`
>>> python frame_extract.py

2. Align faces of the frames, with library face-alignment

>>> python face_align.py -h
usage: face_align.py [-h] [--images-path IMAGES_PATH] [--landmarks-path LANDMARKS_PATH] [--boxes-path BOXES_PATH] [--device DEVICE] [--log-path LOG_PATH]

align faces with `https://github.com/1adrianb/face-alignment`

optional arguments:
  -h, --help            show this help message and exit
  --images-path IMAGES_PATH
                        the input frames path
  --landmarks-path LANDMARKS_PATH
                        the output 68 landmarks path
  --boxes-path BOXES_PATH
                        the output bounding boxes path
  --device DEVICE       cpu or gpu cuda device
  --log-path LOG_PATH   logging when there are no faces detected
  
# default for test: this will generate landmarks and bounding boxes in
# `./test/landmarks` and `./test/boxes`
>>> python face_align.py

3. Crop left eye, right eye, mouth ROIs, with code modified from processing tools of [Eye] RT-GENE and [Mouth] LipForensics

>>> python eye_mouth_crop.py -h
usage: eye_mouth_crop.py [-h] [--images-path IMAGES_PATH] [--landmarks-path LANDMARKS_PATH] [--boxes-path BOXES_PATH] [--eye-width EYE_WIDTH] [--eye-height EYE_HEIGHT]
                         [--face-roi-width FACE_ROI_WIDTH] [--face-roi-height FACE_ROI_HEIGHT] [--left-eye-path LEFT_EYE_PATH] [--right-eye-path RIGHT_EYE_PATH]
                         [--mean-face MEAN_FACE] [--mouth-width MOUTH_WIDTH] [--mouth-height MOUTH_HEIGHT] [--start-idx START_IDX] [--stop-idx STOP_IDX]
                         [--window-margin WINDOW_MARGIN] [--mouth-path MOUTH_PATH]

crop eye and mouth regions

optional arguments:
  -h, --help            show this help message and exit
  --images-path IMAGES_PATH
                        [COMMON] the input frames path
  --landmarks-path LANDMARKS_PATH
                        [COMMON] the input 68 landmarks path
  --boxes-path BOXES_PATH
                        [EYE] the input bounding boxes path
  --eye-width EYE_WIDTH
                        [EYE] width of cropped eye ROIs
  --eye-height EYE_HEIGHT
                        [EYE] height of cropped eye ROIs
  --face-roi-width FACE_ROI_WIDTH
                        [EYE] maximize this argument until there is a warning message
  --face-roi-height FACE_ROI_HEIGHT
                        [EYE] maximize this argument until there is a warning message
  --left-eye-path LEFT_EYE_PATH
                        [EYE] the output left eye images path
  --right-eye-path RIGHT_EYE_PATH
                        [EYE] the output right eye images path
  --mean-face MEAN_FACE
                        [MOUTH] mean face pathname
  --mouth-width MOUTH_WIDTH
                        [MOUTH] width of cropped mouth ROIs
  --mouth-height MOUTH_HEIGHT
                        [MOUTH] height of cropped mouth ROIs
  --start-idx START_IDX
                        [MOUTH] start of landmark index for mouth
  --stop-idx STOP_IDX   [MOUTH] end of landmark index for mouth
  --window-margin WINDOW_MARGIN
                        [MOUTH] window margin for smoothed_landmarks
  --mouth-path MOUTH_PATH
                        [MOUTH] the output mouth images path

# default for test: this will generate the final cropped left eye,
# right eye, and mouth images in `./test/left_eye`, `./test/right_eye`
# , and `./test/mouth`
>>> python eye_mouth_crop.py
  • Note that the argument --face-roi-width and --face-roi-height should be maximized until there is a printed warning.

License

GPL-3.0 License

Reference

[1] Bulat, Adrian, and Georgios Tzimiropoulos. "How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks)." Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2017. GitHub: https://github.com/1adrianb/face-alignment

[2] Fischer, Tobias, Hyung Jin Chang, and Yiannis Demiris. "Rt-gene: Real-time eye gaze estimation in natural environments." Proceedings of the European Conference on Computer Vision (ECCV). 2018. GitHub: https://github.com/Tobias-Fischer/rt_gene

[3] Haliassos, Alexandros, et al. "Lips Don't Lie: A Generalisable and Robust Approach To Face Forgery Detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. GitHub: https://github.com/ahaliassos/LipForensics/

Owner
Zi-Han Liu
Senior @ SJTU
Zi-Han Liu
Unofficial pytorch implementation for Self-critical Sequence Training for Image Captioning. and others.

An Image Captioning codebase This is a codebase for image captioning research. It supports: Self critical training from Self-critical Sequence Trainin

Ruotian(RT) Luo 906 Jan 03, 2023
Public repo for the ICCV2021-CVAMD paper "Is it Time to Replace CNNs with Transformers for Medical Images?"

Is it Time to Replace CNNs with Transformers for Medical Images? Accepted at ICCV-2021: Workshop on Computer Vision for Automated Medical Diagnosis (C

Christos Matsoukas 80 Dec 27, 2022
This repository compare a selfie with images from identity documents and response if the selfie match.

aws-rekognition-facecompare This repository compare a selfie with images from identity documents and response if the selfie match. This code was made

1 Jan 27, 2022
All supplementary material used by me while TA-ing CS3244: Machine Learning

CS3244-Tutorial-Material All supplementary material used by me while TA-ing CS3244: Machine Learning at NUS School of Computing. What is this? I teach

Rishabh Anand 18 Sep 23, 2022
RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation (CIKM'17)

RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation This is the implementation of RATE: Overcoming Noise and Spar

Yu Zhang 5 Feb 10, 2022
This repository is an implementation of paper : Improving the Training of Graph Neural Networks with Consistency Regularization

CRGNN Paper : Improving the Training of Graph Neural Networks with Consistency Regularization Environments Implementing environment: GeForce RTX™ 3090

THUDM 28 Dec 09, 2022
FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics

FusionNet_Pytorch FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics Requirements Pytorch 0.1.11 Pyt

Choi Gunho 102 Dec 13, 2022
TensorFlow Implementation of "Show, Attend and Tell"

Show, Attend and Tell Update (December 2, 2016) TensorFlow implementation of Show, Attend and Tell: Neural Image Caption Generation with Visual Attent

Yunjey Choi 902 Nov 29, 2022
Dynamic hair modeling from monocular videos using deep neural networks

Dynamic Hair Modeling The source code of the networks for our paper "Dynamic hair modeling from monocular videos using deep neural networks" (SIGGRAPH

53 Oct 18, 2022
Official PyTorch implementation of "Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning" (ICCV2021 Oral)

MeTAL - Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning (ICCV2021 Oral) Sungyong Baik, Janghoon Choi, Heewon Kim, Dohee Cho, Jaes

Sungyong Baik 44 Dec 29, 2022
Build and run Docker containers leveraging NVIDIA GPUs

NVIDIA Container Toolkit Introduction The NVIDIA Container Toolkit allows users to build and run GPU accelerated Docker containers. The toolkit includ

NVIDIA Corporation 15.6k Jan 01, 2023
Discord Multi Tool that focuses on design and easy usage

Multi-Tool-v1.0 Discord Multi Tool that focuses on design and easy usage Delete webhook Block all friends Spam webhook Modify webhook Webhook info Tok

Lodi#0001 24 May 23, 2022
Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

Tensor Component Analysis for Interpreting the Latent Space of GANs [ paper | project page ] Code to reproduce the results in the paper "Tensor Compon

James Oldfield 4 Jun 17, 2022
Implementation of OmniNet, Omnidirectional Representations from Transformers, in Pytorch

Omninet - Pytorch Implementation of OmniNet, Omnidirectional Representations from Transformers, in Pytorch. The authors propose that we should be atte

Phil Wang 48 Nov 21, 2022
Implementation of GGB color space

GGB Color Space This package is implementation of GGB color space from Development of a Robust Algorithm for Detection of Nuclei and Classification of

Resha Dwika Hefni Al-Fahsi 2 Oct 06, 2021
Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020)

Causality In Traffic Accident (Under Construction) Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020) Overview Data Prepa

Tackgeun 21 Nov 20, 2022
Deep Reinforced Attention Regression for Partial Sketch Based Image Retrieval.

DARP-SBIR Intro This repository contains the source code implementation for ICDM submission paper Deep Reinforced Attention Regression for Partial Ske

2 Jan 09, 2022
Forecasting with Gradient Boosted Time Series Decomposition

ThymeBoost ThymeBoost combines time series decomposition with gradient boosting to provide a flexible mix-and-match time series framework for spicy fo

131 Jan 08, 2023
Pytorch implementation of COIN, a framework for compression with implicit neural representations 🌸

COIN 🌟 This repo contains a Pytorch implementation of COIN: COmpression with Implicit Neural representations, including code to reproduce all experim

Emilien Dupont 104 Dec 14, 2022
Official code for "Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021".

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021. Introduction We proposed a novel model training paradi

Lucas 103 Dec 14, 2022