Gan Development Series II (pggan, Singan)

GAN The development of series 2 （PGGAN、SinGAN）

We have already introduced it in the previous article GAN The introduction to generating countermeasure networks and some GAN series , In the following album will continue to introduce some of the more classic GAN.

​ ​GAN Introduction to generating countermeasure network ​​

​ ​GAN The development of the series one （CGAN、DCGAN、WGAN、WGAN-GP、LSGAN、BEGAN）​​

One 、PGGAN Incremental growth GAN
The paper ：《Progressive Growing of GANs for Improved Quality, Stability, and Variation》
Address of thesis ：https://arxiv.org/pdf/1710.10196.pdf
Code address ：tkarras/progressive_growing_of_gans

When the resolution of the generated image is very high, the discriminator can easily recognize that the image generated by the generator is false , It makes the generator hard to train , In the previous article DCGAN and WGAN It can only generate 64x64 Image , No matter how big the details are, the loss is serious .PGGAN A new way of training has been introduced , from 4x4 The generated image of begins , until 1024x1024 Face , Let the generator and discriminator grow gradually , Start with low resolution images , Gradually adding new layers makes the network model more complex to learn better details , This method can not only accelerate training, but also make training more stable . This article creates a higher quality version of CELEBA Data sets , Allows output resolution up to 1024 × 1024 Pixels .
PGGAN The progressive approach allows training to first discover the distribution of large-scale structures , Then gradually shift the focus to better scale details , Instead of having to learn all the scales at the same time . Moreover, the generator and discriminator compete with each other to promote the training of both , With GANs The gradual growth of the network , Most of the iterations are done at lower resolution , This reduces the training time .

there 4x4 Refers to the operation of the corresponding size picture , The size of the generator is 4x4 Image , among Reals It refers to the same processed as 4x4 The face image of , The structure of the discriminator is symmetrical to the generator , When the input 800k After a real picture, stop training and save the parameters . And then add the next layer , Because at this time, the number of the final output channels of the generated network is not necessarily 3, So we need to toRGB Convert it to RGB Three channels , For specific operation, use 1x1 Convolution kernel for convolution operation ,fromRGB On the contrary .
In order to prevent the new layer from having a great impact on the original network , When doubling the resolution of the generator and discriminator, a new layer is added smoothly . The figure below explains how to get from 16 × 16 The pixel image is converted to 32 × 32 Pixel image .

The detailed network structure is as follows ：

PGGAN The main advantage is that it can generate high quality samples

PS： I will gradually increase the way of training into UNet The model did some simple experiments , Interested friends can click ：

The paper ：《PGU-net+: Progressive Growing of U-net+ for Automated Cervical Nuclei Segmentation》

Address of thesis ：https://arxiv.org/abs/1911.01062

Code address ：https://github.com/Minerva-J/PGU-net-Model

Two 、SinGAN
The paper 《SinGAN：Learning a Generative Model from a Single Natural Image》
Address of thesis ：https://arxiv.org/abs/1905.01164
Code ：https://github.com/tamarott/SinGAN
More shows ：https://youtu.be/xk8bWLZk4DU

SinGAN Training on only one image , This image is both a training sample and a test sample ,SinGAN It's a non conditional （ Based on random noise ） The generative confrontation model of , Using the full convolution of multi-scale pyramid structure GAN To extract the internal distribution information of the image , Generate high quality with the same visual content 、 Variable samples , Every GAN Responsible for capturing the distribution of images at different scales , A variety of image processing tasks can be applied SinGAN, Such as image drawing 、 edit 、 The fusion , Super resolution reconstruction and animation .

In order to capture the global properties of the shape and position of the target in the image （ Like the sky at the top , The ground is at the bottom ）, And fine details and texture information ,SinGAN Contains a hierarchical structure of patch-GANs（ Markov discriminator ）, Each discriminator is responsible for capturing x Distribution on different scales ,

The loss function consists of resistance loss and reconstruction loss , Against loss is the game between generator and discriminator ,

The reconfiguration loss is

Generate the image ：
1、 Multi scale in the test phase
From N Zebras generated on scales , There are lots of legs . But from the N-1 The scale starts , The generated samples are very real , It's the same with the big trees , And more about details .

2、 Multi scale in the training phase
When the scale is richer , Can capture the global structure .