(论文翻译]未配对Image-To-Image翻译使用Cycle-Consistent敌对的网络

[论文地址][代码][ICCV 17]

Only the method part is translated

III. Formulation

Our target is given training samples${\left\{x_i\right\}}_{i=1}^N\in X$和{ ${\left\{x_j\right\}}_{j=1}^M\in X$Learn between the two domains$X$和$Y$的映射函数.如图3(a)所示,我们的模型包括两个映射$G:X\to Y$和$F:Y\to X$.此外,We introduce two adversarial discriminators$D_X$和$D_Y$,其中$D_X$旨在区分图像$\{x\}$and translated images$\{F(y)\}$;同样,$D_Y$旨在区分$\{y\}$和$\{G(x)\}$.Our goals consist of two kinds：对抗性损失(Adversarial Loss),Used to match the resulting image distribution to the data distribution of the target domain;以及循环一致性损失(Cycle Consistency Loss
),to prevent learned mappings$G$和$F$相互矛盾.

Adversarial Loss

We apply adversarial loss to two mapping functions.对于映射函数$G:X\to Y$and its discriminator$D_Y$,We express the goal as ：$$\begin{gathered} {\mathcal{L}}_{\text{GAN}}(G,D_Y,X,Y)={\mathbb{E}}_{y\sim p_{\text{data}}(y)}[\log D_Y(y)] \\ +{\mathbb{E}}_{x\sim p_{\text{data}}(x)}[\log (1-D_Y(G(x))] \end{gathered}$$ 其中$G$Trying to generate with fields$Y$The image is similar to the image$G(x)$,而$D_Y$Aims to distinguish translated samples$G(x)$and real samples$Y$.我们为映射函数$F:Y\to X$and its discriminator$D_X$也引入了类似的对抗性损失：即$L_{GAN}(F,D_X,Y,X)$.

Cycle Consistency Loss

理论上,对抗性训练可以学习映射$G$和$F$,Generated with the target domain$Y$和$X$distribute the same output(严格来说,这需要G和F是随机的函数).然而,with sufficient capacity,The network can map the same set of input images to any random arrangement of images in the target domain,Any of these learned mappings can lead to an output distribution that matches the target distribution.为了进一步减少可能的映射函数的空间,We believe that the learned mapping function should be cycle-consistent：如图3b所示,对于来自域$X$的每个图像$x$,The image translation loop should be able to$x$带回原始图像,即$x\to G(x)\to F(G(x))\approx x$.We call this forward cycle consistency.同样,如图3c所示,对于来自域$Y$的每个图像$y$,$G$和$F$Backward circular consistency should also be satisfied：$y\to F(y)\to G(F(y))\approx y$.We can incentivize this behavior with a cycle consistency loss.$$\begin{gathered} {\mathcal{L}}_{\text{cyc}}(G,F)={\mathbb{E}}_{x\sim p_{\text{data}}(x)}[\Vert F(G(x))-x{\Vert }_1] \\ +{\mathbb{E}}_{y\sim p_{\text{data}}((y)}[\Vert G(F(y))-y{\Vert }_1]. \end{gathered}$$ 在初步实验中,我们还尝试用$F(G(x))$和$x$之间以及$G(F(y))$和$y$The adversarial loss between is replaced in this lossL1准则,但没有观察到性能的改善.Behavior caused by cycle consistency loss can be found in arXivversion observed.

Full Objective

Our overall objective function is ：$$\begin{gathered} \mathcal{L}(G,F,D_X,D_Y)={\mathcal{L}}_{\text{GAN}}(G,D_Y,X,Y) \\ +{\mathcal{L}}_{\text{GAN}}(F,D_X,Y,X) \\ +\lambda {\mathcal{L}}_{\text{cyc}}(G,F) \end{gathered}$$ 其中$\lambda$控制两个目标的相对重要性.我们的目标是解决：$$G^{\ast },F^{\ast }=\arg \underset{G,F}{\min }\underset{D_x,D_Y}{\max }\mathcal{L}(G,F,D_X,D_Y)$$ 请注意,Our model can be seen as training two"自动编码器"：We will use an autoencoder$F\circ G:X\to X$与另一个$G\circ F:Y\to Y$共同学习.然而,这些自动编码器都有特殊的内部结构：They map images to themselves through an intermediate representation,This intermediate representation is the translation of the image in another domain.Such a setup can also be seen as"对抗性自动编码器"的一个特例,It uses an adversarial loss to train the bottleneck layer of the autoencoder to match an arbitrary target distribution.在我们的例子中,$X\to X$The target distribution of the autoencoder is the domain$Y$的分布.在第5.1.3节中,We compare our method with full target subtraction,and shown by experience,These two goals play a key role in obtaining high-quality results.