Deep learning series 48:deepfaker

1. Effect display

In a general way ,deepfake It can be divided into the following four categories ： repeat （reenactment）, Replace (replace), edit (editing) And synthesis (synthesis).
For face 4 The categories are shown below ：

The difference between reproduction and replacement is ：
The reappearance is equivalent to Angela baby playing , But the facial expression is interpreted according to the double ; Replacement is equivalent to a stand in , Then change to Angela baby's face .

2. Basic technology

The core technology ： The first one is to use an automatic encoder and then manipulate the middle latent Vector to achieve the goal ; The second is to use GAN Migration .
Each task requires some additional modules . We use First Order Motion Model for Image Animation For example , The task it wants to accomplish is image animation, Enter a source image （source image） And a drive video (driving video), The output is a video , The main character is the source image , Action is driving the action in the video . As shown below , The source image usually contains a subject , The driving video contains a series of actions .

The model will observe the frame pairs extracted from the same video , It will learn to encode actions as a combination of action specific key displacement and local affine transformation .
The whole model is divided into motion estimation module and image generation module . In the motion estimation module , The model separates the appearance and motion information of the object by self supervised learning , And feature representation .
In the image generation module , The model models the occlusion that occurs during the target's motion , Then extract the appearance information from the given Celebrity Image , Combined with previously obtained feature representation , Video synthesis .
Note that the generation module is a AE, Motion information manipulates hidden space d, Then generate a new image sequence .

3. DeepFaceLab course

Most face changing videos are made with this project .git The address is ：https://github.com/iperov/DeepFaceLab