Gan model architecture in mm

mmgeneration Medium GAN The architecture of includes , Enter a real image , If it's a condition gan Words , What you input is a real image and the corresponding label ,

1. First, train the discriminator , The gradient in the discriminator is saved during training , And the discriminator optimizer zero_grad(),gan There is an optimizer in the generator and the discriminator , It's actually two relatively independent training processes .

2. When training the discriminator, the generator is also used to generate a false sample , Note that the generator at this time is not trained , Just generate a fake sample , In terms of gan when , There is also a batch_accumulation_step Steps for , It will train several more times during generator training .

3. Use a discriminator to distinguish between true samples and false samples , Calculate the loss of discriminator training , natural ganloss And the corresponding auxiliary losses , Put it in disc_auxiliary_loss in , Feeling gan There are not many directions that can be expanded in , Architecturally speaking transformers perhaps self-attention And so on. , In theory, it is basically a loss function ,l Some operations on constraints , So at this time gen_auxiliary_loss Is a different point . The discriminator's loss It is essentially a two category , Negative sample input , The label given is 0, The label of positive sample input is 1.

4. After calculating the loss ,loss_disc.bachward() Back propagation ,optimizer['disciminator'].step() Gradient update , Here the discriminator is trained .

5. Conduct 5-8 After the discriminator training of the wheel, start the steering generator training , In fact, the gradient is cleared before each training , It is equivalent to not accumulating gradients in each round , First, the gradient of the discriminator is not saved , Notice when training the discriminator at the beginning , The gradient of the discriminator is preserved , Then the gradient of the generator is zero_grad().

6. Training generator , The input of the generator is still noise , After getting the image of the generator , Perform a discriminator decision , The last layer of the discriminator is usually a Linear(n,1) The layer , That is, the output is N,1 Dimensional , Input to gen_loss in , here gen_loss The middle core is also an auxiliary loss gen_auxiliary_loss in , Some may also be added here l Constraints and so on .

7. After calculating the loss ,loss_gen.backward() Back propagation ,optimizer['generator'].step() Gradient update , Generator training complete .

The figure above shows the structure of a typical generator and discriminator , In generator , We need a module that converts noise vectors into two-dimensional features , That is to say noise2feat block. Next, several upsampling blocks are needed to convert the low-resolution features into high-resolution features , stay DCGAN in , We use transposed convolution To achieve . Last , Need one to_rgb Block to map the number of channels of the feature graph to 3 passageway , So as to generate pictures . The discriminator is actually an inversion of the generator , We need to pass img2feat And a large number of down sampled blocks will continuously reduce the resolution of the feature map , Finally delivered to decision head, To judge the current input image .