cyclegan:unpaired image-to-image translation using cycle-consistent adversarial network

CycleGAN Reading and translation of papers , Unsupervised style migration - You know 2018-10-10 First edition , The thesis is not short , Please choose the important part to read 2018-11-02 Discussion with the comments section 2018-11-25 Grammar error minor modification 2019-08-14 Comment area reply , Other developments in unsupervised style migration 2019-08-26 Revise the translation of the thesis according to the suggestions in the comments section …https://zhuanlan.zhihu.com/p/45394148cyclegan The core of it is unpaired image, If it is pair Data words , Direct use pix2pix that will do , But unpaired data is used cyclegan,cyclegan and biggan equally , The article is easy to understand .cyclegan The core of is from X To Y, The mapping space for migration between two domains is very large , The author uses cycle consistent loss Constraints have been made. , Intuition is right X adopt G Generated Y And then through F Turn around , use l1 loss To supervise ,Y The same goes for the side loss , Usually cyclegan There are four losses , Two gan Of , Two cycle loss.

1.abstract:

        In this paper, we propose a method to learn the image from the source domain without pairing examples X To the target domain Y Methods , Learn a mapping G：X->Y, come from G(x) Image distribution and distribution Y There is no difference between , Because the mapping is highly under-constrained, Using a quasi mapping F：Y->X, And introduced cycle consistency loss mandatory F(G(X))==Y.

2.Introduction

        We seek an algorithm that can transform between learning domains without pairing input and output examples , We assume that there are some potential relationships between domains , For example, they are two different renderings in the same scene , We try to learn this relationship . Theoretically ,G（x） What I learned y The distribution of is to match the prior distribution Y Of , Usually this requires G Is random , However , Such a transformation does not guarantee a single input x And output are matched in a meaningful way , There are infinite mappings G Will be in y Produce the same distribution on , So there is a pattern crash . here x From tradition Gan Is the same distribution , from pix2pix The point of view is the different distribution of paired data .

        We proposed cycle consistent, If you translate a sentence from English to French , Should also be able to return from French to English , Mathematically ,G and F Should be reciprocal , And both mappings should be bijective . Training at the same time G and F, And add cycle consistency loss Make sure F(G(x))=x and G(F(y))=y. Combine this loss with the frame loss , The goal of unpaired image to image conversion can be achieved .

 2.related work

        Gans、image-to-image translation、unpaired image-to-image translation、cycle consistency、neural style transfer

3.formulation

The above figure is the core of the text , chart a It's architecture ,G and F It's a double shot , Except for the normal GAN Outside of the generator and discriminator , here X The city generator becomes Y The distribution of , And then through the generator F take Y Mapping back to X, Mapping back to X Distribution and raw input x adopt cycle consistency loss To be consistent ,y The same is true for domain graphs , Note here that the input is a graph of two fields that are not paired ,cycle consistency loss It is also two losses added on the original basis , In theory gan There are four loss functions , A discriminator and a generator , Two domains of cycle loss, There will be two more in some scenes id loss. This one is quite exquisite , according to Gan The idea of , from X To Y Migration ,X Most of them are Gaussian distribution or uniform distribution , Generate a Y The distribution of , however cyclegan Is a potential relationship of learning , So light from X To Y Your study will promote X Direct conversion Y Without retaining X The nature of , So it's reserved cycle loss This ability to learn bijection .

3.1 Adversarial loss

3.2 cycle consistence loss

        from X To Y The mapping space of must be constrained , The mapping space is controlled by forward and backward cyclic consistency .

3.3 full objective

The author also did an experiment , In theory, a forward circulation loss should be enough , But cycle losses are carried out in only one direction , Not enough to constrain this space .

Among them in application Also mentioned in idt loss, I think mm There are... In the official id loss And no addition id loss It's all calculated ,id loss, prevent input and output Between color composition Too much , Avoid excessive migration .

4.Implementation

The following is the author's implementation details and experimental part , I won't go into that , The main thing is to understand Gan The core idea of .