Detailed introduction and application of GaN (comprehensive and complete)

GAN And Its Application

introduction

---

---

Application

---

Posture guides human figure generation

Through additional input of posture , We can convert images into different poses . for example , The image in the upper right corner is the basic pose , In the lower right corner is the generated image .

---

The following optimization result column is the generated image .

---

The design consists of a two-stage image generator and a frequency discriminator . Generator uses metadata （ posture ） And the original image to reconstruct the image . The discriminator uses the original image as CGAN Design part of label input .

---

CycleGAN

Cross domain name transfer is likely to become the first batch of commercial applications .GANs Take images from one field （ Such as the real scenery ） Switch to another field （ Monet painting or van Gogh ）.

---

for example , It can convert pictures between zebras and horses .

---

Cyclegan Two networks are built G and F To build images from one domain to another and vice versa . It uses a discriminator d To criticize how good the generated image is . for example ,G Transform real images into Van Gogh style paintings , also DY Used to distinguish whether the image is real or generated .

Domain A To domain B：------------------------>

We are in the reverse domain B Domain A Repeat the process ：

---

PixelDTGAN

Recommending products based on celebrity pictures has become a hot topic in fashion blogs and e-commerce .Pixeldtgan The function of is to create clothing images and styles from images .

---

---

Super resolution

Create a super-resolution image from a low resolution . This is a GAN Showing very impressive results , It is also an area with direct commercial possibilities .

---

And many GAN Similar in design , It is composed of multiple convolution layers 、 Batch of standardized 、 senior relu And jump connection .

---

PGGAN

Progressive GAN It may be the first to show commercial image quality GAN One of . The following is by GAN Created 1024×1024 Celebrity Image .

---

It adopts a divide and rule strategy , Make training more feasible . The convolution layer is trained again and again to build 2 Multiple resolution image .

---

stay 9 In three stages , Generate 1024×1024 Images .

---

High resolution image synthesis

It should be noted that this is not image segmentation , Instead, images are generated from semantic maps . Because collecting samples is very expensive , We use the generated data to supplement the training data set , To reduce development costs . When training autonomous vehicle, it can automatically generate video , Instead of seeing them cruising around , This brings convenience to our life .

Network design :

---

Text to image （StackGAN）

Text to image is domain transfer GAN One of the early applications of . such as , We can generate multiple images that match the description by inputting a sentence .

---

Text to image synthesis

Another general implementation ：

---

Face synthesis

Synthetic surfaces under different postures ： Use a single input image , We can create faces from different perspectives . for example , We can use it to transform images that are easier for face recognition .

---

Image restoration

Decades ago , Image restoration has always been an important topic .gan Can be used to repair images and create “ Content ” Fill in the missing part .

---

Learn joint distribution

Use facial characters P（ golden hair , women , smile , wear glasses ）,P（ Brown , men , smile , No glasses ） Wait for different combinations to create GAN It's very unrealistic . The curse of dimension makes GAN The number of is increasing exponentially . But we can learn individual data distributions and combine them to form different distributions , That is, different attribute combinations .

---

DiscoGAN

DiscoGAN Provides a matching style ： Many potential applications .DiscoGAN Learn cross domain relationships without labels or pairing . for example , It successfully converts the style （ Or pattern ） From a domain （ handbag ） Transfer to another domain （ shoes ）.

DiscoGAN and cyclegan It is very similar in network design .

---

---

Pix2Pix

PIX2PIx It is a kind of image to image translation , Across domains Gan Are often cited in the papers . for example , It can convert satellite images into maps （ Bottom left corner of picture ）.

---

DTN

Create emoticons from pictures .

---

Texture synthesis

---

image editing (IcGAN)

Rebuild or edit images with specific properties .

---

Aging faces (Age-cGAN)

---

Create animated characters

as everyone knows , Game development and animation production costs are high , And hired many production artists to complete relatively routine tasks . But through GAN You can automatically generate animated characters and color them .

Use Generative Adversarial Networks Create automatic animated personas

---

The generator and discriminator consist of multiple convolution layers 、 Batch standardization and with skip links relu form .

---

Neurophoto editor

Content based image editing ： for example , Extended hair band .

Neurophoto editor

---

Thinning image

object detection

This is the use of gan An application that enhances an existing solution .

---

Image fusion

Blend images together .

Generate 3D objects

This is the use of gan An article often cited when creating 3D objects .

---

The production of music

GaN It can be applied to non image fields , Compose music like .

---

Medical care （ Anomaly detection ）

GAN It can also be extended to other industries , For example, tumor detection in medicine .