Paddlepaddle paper reproduction course biggan learning experience

PaddlePaddle Thesis reproduction course -BigGan The learning

The course of paper reproduction of flying oars ←（ The official link ）

Course description

In order to help you better carry out scientific research work such as paper publishing , Deep practice and deep learning , Baidu specially launched the global summit in-depth learning thesis recovery camp . It is taught jointly by Baidu senior algorithm engineer and senior researcher of Chinese Academy of Sciences ,28 Heaven's hand will show you in person 1 Papers , Master the whole process of paper reproduction .

The goal is

1. Deeply master the accuracy of the paper 、 Model structures, 、 Model tuning and other in-depth learning of hard core skills .

2. Master GAN、 Video classification technology direction , Successfully reproduce at least 1 A top conference academic paper .

Related links

https://aistudio.baidu.com/aistudio/education/group/info/1340

The paper ：Large Scale GAN Training For High Fidelity Natural Image Synthesis←（ The paper PDF）

Write it at the front

Choose this paper to read it carefully and share it , Just because of its effect .
（ Cut from this Paper Page.1）

At the beginning of the paper, the amazing effects and results are given .

Our modifications lead to models which set the new state of the art in class-conditional image synthesis. When trained on ImageNet at 128×128 resolution, our models (BigGANs) achieve an Inception Score (IS) of 166.5 and Frechet Inception Distance (FID) of 7.4, improving over the previous best IS of 52.52 and FID of 18.65.¹

from FID：18.65 To 7.4,IS：52.52 To 166.5, This huge leap is really exciting .（ Although this is a high score achieved by sacrificing diversity and focusing on realism generation , Its fidelity is really hard to distinguish with the naked eye .）
From its rendering , Except for the obvious powerlessness of the characters , Other animals 、 scenery 、 The creation of buildings and other scenes has already had a good effect .

The main points of the paper

The author clearly describes its purpose to enhance Gan Three directions of the generated image and the real image ：

1. GAN The scale of
2. “ Cutting skills ”
3. Sacrifice a certain degree of stability

The paper GAN Network base ：SA-GAN

GAN scale

Here the author chooses to expand GAN Network training batch size and channel number （ Add parameters ）
1. Training batch size ： Discovery to expand to 8 times ,IS Can increase 46%.
2. The channel number ： Expand to the original 1.5 times ,IS Can increase 21%.
（ Cut from this Paper Page.3）

“ Cutting skills ”

The author is describing “ Cutting skills ” Gives it a qualitative role “TRADING OFF VARIETY AND FIDELITY WITH THE TRUNCATION TRICK”—— Balance of diversity and fineness .
Mainly for input to G Generator's Z The signal goes on “ Choice ” Adoption of , Set a certain Threshold.
（ Cut from this Paper Page.4）

For larger networks, there may be problems that cannot be solved “ Cutting skills ”, The author introduces the orthogonal regularity which he has polished .

$$R_{\beta }\left(W\right)=\beta ||W^{T}W\odot \left(1-I)\right){]}_F^2$$
among 1 Indicates that the elements are all 1 Matrix ,β=10e-4,W Is the parameter vector .

Sacrifice a certain degree of stability

The author discovered from experience that large-scale GANs Characteristic instability , Their characteristics are described according to experience . Use the insights from this analysis , It proves that the combination of new technology and existing technology can reduce these instabilities , But complete training stability can only be achieved at a huge performance cost .
Given the larger scale GAN The number of iterations required for training to achieve better results will also decrease , The author chooses to stop training before the model collapses .
（ Cut from this Paper Page.5）

---

1. LARGE SCALE GAN TRAINING FOR HIGH FIDELITY NATURAL IMAGE SYNTHESIS Andrew Brock, Jeff Donahue, Karen Simonyan︎