Detailed explanation of diffusion model

1 introduction

  In the last article 《 Depth generation model based on flow 》 The theory and method of flow generation model are introduced in detail . So far, , be based on GAN Generate models , be based on VAE The generation model of , And based on flow All of them can generate high-quality samples , But each method has its limitations .GAN In the process of confrontation training, there will be problems of mode collapse and unstable training ;VAE It depends heavily on the target loss function ; The flow model must use a special framework to build reversible transformations . This paper mainly introduces the diffusion model , Its inspiration comes from non-equilibrium thermodynamics . They define Markov chains of diffusion steps , Slowly add random noise to the data , Then learn the reverse diffusion process to construct the required data samples from the noise . And VAE Or different flow models , The diffusion model is learned through a fixed process , And the hidden variables in the middle have high dimensions with the original data .

• advantage ： The diffusion model is both easy to analyze and flexible . Be aware that manageability and flexibility are two conflicting goals in Generative modeling . Easy to handle models can be used to analyze, evaluate and fit data , But they cannot easily describe the structure of rich data sets . The flexible model can fit any structure in the data , But evaluate from these models 、 The cost of training or sampling can be high .
• shortcoming ： The diffusion model relies on a long Markov diffusion step chain to generate samples , Therefore, the cost in terms of time and calculation will be very high . At present, new methods have been proposed to make the process faster , But the overall process of sampling is still better than GAN slow .
  

2 Forward diffusion process

  Given the distribution from real data ${\mathbf{x}}_0\sim q(\mathbf{x})$ Data points sampled in , In a forward diffusion process , stay $T$ Step by step, add a small amount of Gaussian noise to the sample , Thus, a series of noise samples are generated ${\mathbf{x}}_1,\cdots ,{\mathbf{x}}_T$, Its step length is planned by variance $\{{\beta }_t\in (0,1){\}}_{t=1}^T$ To control , Then there are $$q({\mathbf{x}}_t|{\mathbf{x}}_{t-1})=\mathcal{N}({\mathbf{x}}_t;\sqrt{1-\beta }{\mathbf{x}}_{t-1},{\beta }_t\mathbf{I})q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)=\prod _{t=1}^{T}q({\mathbf{x}}_t|{\mathbf{x}}_{t-1})$$ During the diffusion process , With the time step $t$ The increase of , Data samples ${\mathbf{x}}_0$ Gradually lose its distinguishing features . Final , When $T\to \infty$, ${\mathbf{x}}_T$ Equivalent to isotropic Gaussian distribution （ Isotropic Gaussian distribution is called spherical Gaussian distribution , In particular, it refers to the multi-dimensional Gaussian distribution with the same variance in all directions , The covariance is a positive real number multiplied by the identity matrix ）.

A good feature of the above process is that the reparameterization technique can be used in a closed form at any time step $t$ Yes ${\mathbf{x}}_t$ sampling . Make ${\alpha }_t=1-{\beta }_t$ and ${\bar{\alpha }}_t={\prod }_{i=1}^T{\alpha }_i$, Then there are ：$$\begin{array}{rl}{\mathbf{x}}_t& =\sqrt{{\alpha }_t}{\mathbf{x}}_{t-1}+\sqrt{1-{\alpha }_t}{\mathbf{z}}_{t-1}\\ & =\sqrt{{\alpha }_t{\alpha }_{t-1}}{\mathbf{x}}_{t-2}+\sqrt{1-{\alpha }_t{\alpha }_{t-1}}{\bar{\mathbf{z}}}_{t-2}\\ & =\cdots \\ & =\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{z}\\ q({\mathbf{x}}_t|{\mathbf{x}}_0)& =\mathcal{N}({\mathbf{x}}_t;\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0,(1-{\bar{\alpha }}_t)\mathbf{I})\end{array}$$ among ${\mathbf{z}}_{t-1},{\mathbf{z}}_{t-2},\cdots \sim \mathcal{N}(0,\mathbf{I})$,${\bar{\mathbf{z}}}_{t-2}$ Fuse two Gaussian distributions . When merging two with different variances $\mathcal{N}(0,{\sigma }_1^2\mathbf{I})$ and $\mathcal{N}(0,{\sigma }_2^2\mathbf{I})$ When the Gaussian distribution of , The new Gaussian distribution obtained is $\mathcal{N}(0,({\sigma }_1^2,{\sigma }_2^2)\mathbf{I})$, The combined standard deviation is $$\sqrt{(1-{\alpha }_t)+{\alpha }_t(1-{\alpha }_{t-1})}=\sqrt{1-{\alpha }_t{\alpha }_{t-1}}$$ Usually , The larger the noise is, the larger the update step will be , Then there are ${\beta }_1<{\beta }_2\cdots <{\beta }_T$, therefore ${\bar{\alpha }}_1>\cdots >{\bar{\alpha }}_T$.

3 The update process

 Langevin Dynamics is a concept in Physics , For statistical modeling of molecular systems . Combined with random gradient descent , Random gradient Langevin dynamics can only use the gradient in the Markov update chain ${\nabla }_{\mathbf{x}}\log p(\mathbf{x})$ From the probability density $p(\mathbf{x})$ Generated samples ：$${\mathbf{x}}_t={\mathbf{x}}_{t-1}+\frac{ϵ}{2}{\nabla }_{\mathbf{x}}\log p({\mathbf{x}}_{t-1})+\sqrt{ϵ}{\mathbf{z}}_t,{\mathbf{z}}_t\sim \mathcal{N}(0,\mathbf{I})$$ among $ϵ$ Step length . When $T\to \infty$ when ,$ϵ\to 0$,$\mathbf{x}$_T Is equal to the true probability density $p(\mathbf{x})$. With the standard SGD comparison , Random gradient Langevin Dynamics injects Gaussian noise into parameter update , To avoid falling into the local minimum .

4 Reverse diffusion process

  If the above process is reversed and the probability distribution $q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)$ Sampling in , Can be input from Gaussian noise ${\mathbf{x}}_T\sim \mathcal{N}(0,\mathbf{I})$ Reconstruct the real sample . It should be noted that if ${\beta }_t$ Small enough ,$q({\mathbf{x}}_{t-1},{\mathbf{x}}_t)$ It will also be Gaussian distribution . But this needs to be estimated using the entire data set , So we need to learn a model $p_{\theta }$ To approximate these conditional probabilities , In order to carry out the reverse diffusion process $$p_{\theta }({\mathbf{x}}_{0:T})=p({\mathbf{x}}_T)\prod _{t=1}^T{p}_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)=\mathcal{N}({\mathbf{x}}_{t-1};{\mathbf{\mu }}_{\theta }({\mathbf{x}}_t,t),{\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t))$$ When conditions are ${\mathbf{x}}_0$ when , The inverse conditional probability is easy to estimate ：$$q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)=\mathcal{N}({\mathbf{x}}_{t-1};\mathbf{\mu }({\mathbf{x}}_t,{\mathbf{x}}_0),{\tilde{\beta }}_t\mathbf{I})$$ Using Bayesian law, we can get $$\begin{array}{rl}q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)& =q({\mathbf{x}}_t|{\mathbf{x}}_{t-1},{\mathbf{x}}_0)\frac{q({\mathbf{x}}_{t-1}|{\mathbf{x}}_0)}{q({\mathbf{x}}_t|{\mathbf{x}}_0)}\\ & \propto \exp \left[-\frac{1}{2}\left(\frac{({\mathbf{x}}_t-\sqrt{{\alpha }_t}{\mathbf{x}}_{t-1}{)}^2}{{\beta }_t}+\frac{({\mathbf{x}}_{t-1}-\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0{)}^2}{1-{\bar{\alpha }}_{t-1}}-\frac{({\mathbf{x}}_t-\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0{)}^2}{1-{\bar{\alpha }}_t}\right)\right]\\ & =\exp \left[-\frac{1}{2}\left(\frac{{\mathbf{x}}_t^2-2\sqrt{{\alpha }_t}{\mathbf{x}}_t{\mathbf{x}}_{t-1}+{\alpha }_t{\mathbf{x}}_{t-1}^2}{{\beta }_t}+\frac{{\mathbf{x}}_{t-1}^2-2\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0{\mathbf{x}}_{t-1}+{\bar{\alpha }}_{t-1}{\mathbf{x}}_0}{1-{\bar{\alpha }}_{t-1}}-\frac{({\mathbf{x}}_t-\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0{)}^2}{1-{\bar{\alpha }}_t}\right)\right]\\ & =\exp \left[-\frac{1}{2}\left(\left(\frac{{\alpha }_t}{{\beta }_t}+\frac{1}{1-{\bar{\alpha }}_{t-1}}\right){\mathbf{x}}_{t-1}^2-\left(\frac{2\sqrt{{\alpha }_t}}{{\beta }_t}{\mathbf{x}}_t+\frac{2\sqrt{{\bar{\alpha }}_{t-1}}}{1-{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\right){\mathbf{x}}_{t-1}+C({\mathbf{x}}_t,{\mathbf{x}}_0)\right)\right]\end{array}$$ among $C({\mathbf{x}}_t,{\mathbf{x}}_0)$ Function and ${\mathbf{x}}_{t-1}$ irrelevant . According to the standard Gaussian density function , The mean and variance can be parameterized as follows $$\begin{array}{rl}{\tilde{\beta }}_t& =1/\left(\frac{{\alpha }_t}{{\beta }_t}+\frac{1}{1-{\bar{\alpha }}_{t-1}}\right)=1/\left(\frac{{\alpha }_t-{\bar{\alpha }}_t+{\beta }_t}{{\beta }_t(1-{\bar{\alpha }}_{t-1})}\right)=\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}\cdot {\beta }_t\\ {\tilde{\mathbf{\mu }}}_t({\mathbf{x}}_t,{\mathbf{x}}_0)& =\left(\frac{{\sqrt{\alpha }}_t}{{\beta }_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}}{1-{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\right)/\left(\frac{{\alpha }_t}{{\beta }_t}+\frac{1}{1-{\bar{\alpha }}_{t-1}}\right)\\ & =\left(\frac{{\sqrt{\alpha }}_t}{{\beta }_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}}{1-{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\right)\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}\cdot {\beta }_t\\ & =\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}{\beta }_t}}{1-{\bar{\alpha }}_t}{\mathbf{x}}_0\end{array}$$ take ${\mathbf{x}}_0=\frac{1}{\sqrt{{\bar{\alpha }}_t}}({\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}{\mathbf{z}}_t)$ Brought into the above formula is $$\begin{array}{rl}{\tilde{\mathbf{\mu }}}_t& =\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}\frac{1}{\sqrt{{\bar{\alpha }}_t}}({\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}{\mathbf{z}}_t)\\ & =\frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathbf{z}}_t\right)\end{array}$$ This setting is similar to VAE Very similar , Therefore, the variational lower bound can be used to optimize the negative log likelihood , Then there are $$\begin{array}{rl}-\log p_{\theta }({\mathbf{x}}_0)& \le -\log p_{\theta }({\mathbf{x}}_0)+D_{\mathrm{K}\mathrm{L}}(q({\mathbf{x}}_{1:T})|{\mathbf{x}}_0||p_{\theta }({\mathbf{x}}_{1:T}|{\mathbf{x}}_0))\\ & =-\log p_{\theta }({\mathbf{x}}_{\theta })+{\mathbb{E}}_{1:T\sim q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}\left[\log \frac{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{0:T})/p_{\theta }({\mathbf{x}}_0)}\right]\\ & =-\log p_{\theta }({\mathbf{x}}_0)+{\mathbb{E}}_q\left[\log \frac{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{0:T})}+\log p_{\theta }({\mathbf{x}}_0)\right]\\ & ={\mathbb{E}}_q\left[\log \frac{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{0:T})}\right]\\ L_{\mathrm{V}\mathrm{L}\mathrm{B}}& ={\mathbb{E}}_{q({\mathbf{x}}_{0:T})}\left[\log \frac{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{0:T})}\right]\ge -{\mathbb{E}}_{q({\mathbf{x}}_0)}\log p_{\theta }({\mathbf{x}}_0)\end{array}$$ Use Jensen Inequality can easily get the same result . Suppose we want to minimize cross entropy as the learning goal , Then there are $$\begin{array}{rl}{L}_{\mathrm{C}\mathrm{E}}& =-{\mathbb{E}}_{q({\mathbf{x}}_0)}\log p_{\theta }({\mathbf{x}}_0)\\ & =-{\mathbb{E}}_{q({\mathbf{x}}_0)}\log \left(\int p_{\theta }({\mathbf{x}}_{0:T})d{\mathbf{x}}_{1:T}\right)\\ & =-{\mathbb{E}}_{q({\mathbf{x}}_0)}\log \left(\int q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)\frac{p_{\theta }({\mathbf{x}}_{0:T})}{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}d{\mathbf{x}}_{1:T}\right)\\ & =-{\mathbb{E}}_{q({\mathbf{x}}_0)}\log \left({\mathbb{E}}_{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}\frac{p_{\theta }({\mathbf{x}}_{0:T})}{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}\right)\\ & \le -{\mathbb{E}}_{q({\mathbf{x}}_{0:T})}\log \frac{p_{\theta }({\mathbf{x}}_{0:T})}{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}\\ & ={\mathbb{E}}_{q({\mathbf{x}}_{0:T})}\left[\log \frac{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{0:T})}\right]=L_{\mathrm{V}\mathrm{T}\mathrm{B}}\end{array}$$ In order to convert each term in the equation into analytically calculable , The goal can be further rewritten into several KL The combination of divergence and entropy $$\begin{array}{rl}{L}_{\mathrm{T}\mathrm{V}\mathrm{B}}& ={\mathbb{E}}_{q({\mathbf{x}}_{0:T})}\left[\log \frac{q({\mathbf{x}}_{1:T}|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{0:T})}\right]\\ & ={\mathbb{E}}_q\left[\log \frac{{\prod }_{t=1}^{T}q({\mathbf{x}}_t|{\mathbf{x}}_{t-1})}{p_{\theta }({\mathbf{x}}_T){\prod }_{t=1}^T{p}_{\theta }({\mathbf{x}}_{t-1}|p({\mathbf{x}}_t))}\right]\\ & ={\mathbb{E}}_q\left[-\log p_{\theta }({\mathbf{x}}_T)+\sum _{t=1}^T\log \frac{q({\mathbf{x}}_t|{\mathbf{x}}_{t-1})}{p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)}\right]\\ & ={\mathbb{E}}_q\left[-\log p_{\theta }({\mathbf{x}}_T)+\sum _{t=2}^T\log \frac{q({\mathbf{x}}_t|{\mathbf{x}}_{t-1})}{p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)}+\log \frac{q({\mathbf{x}}_1|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_0|{\mathbf{x}}_1)}\right]\\ & ={\mathbb{E}}_q\left[-\log p_{\theta }({\mathbf{x}}_T)+\sum _{t=2}^T\log \left(\frac{q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)}\cdot \frac{q({\mathbf{x}}_t|{\mathbf{x}}_0)}{q({\mathbf{x}}_{t-1}|{\mathbf{x}}_0)}\right)+\log \frac{q({\mathbf{x}}_1|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_0|{\mathbf{x}}_1)}\right]\\ & ={\mathbb{E}}_q\left[-\log p_{\theta }({\mathbf{x}}_T)+\sum _{t=2}^T\log \frac{q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)}+\sum _{t=2}^T\log \frac{q({\mathbf{x}}_t|{\mathbf{x}}_0)}{q({\mathbf{x}}_{t-1}|{\mathbf{x}}_0)}+\log \frac{q({\mathbf{x}}_1|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_0|{\mathbf{x}}_1)}\right]\\ & ={\mathbb{E}}_q\left[-\log p_{\theta }({\mathbf{x}}_T)+\sum _{t=2}^T\log \frac{q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)}+\log \frac{q({\mathbf{x}}_T|{\mathbf{x}}_0)}{q({\mathbf{x}}_1|{\mathbf{x}}_0)}+\log \frac{q({\mathbf{x}}_1|{\mathbf{x}}_0)}{p_{\theta }(\mathbf{x}|{\mathbf{x}}_1)}\right]\\ & ={\mathbb{E}}_q\left[\log \frac{q({\mathbf{x}}_T|{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_T)}+\sum _{t=2}^T\log \frac{q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)}{p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)}-\log p_{\theta }({\mathbf{x}}_0|{\mathbf{x}}_1)\right]\\ & ={\mathbb{E}}_q\left[D_{\mathrm{K}\mathrm{L}}(q({\mathbf{x}}_T|{\mathbf{x}}_0)||p_{\theta }({\mathbf{x}}_T))+\sum _{t=2}^T{D}_{\mathrm{K}\mathrm{L}}(q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)||p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t))-\log p_{\theta }({\mathbf{x}}_0|{\mathbf{x}}_1)\right]\end{array}$$ Mark each component in the lower bound loss of variation as $$\begin{array}{rl}{L}_{\mathrm{V}\mathrm{L}\mathrm{B}}& =L_T+L_{T-1}+\cdots +L_0\\ L_T& =D_{\mathrm{K}\mathrm{L}}(q({\mathbf{x}}_T|{\mathbf{x}}_0)||p_{\theta }({\mathbf{x}}_T))\\ L_t& =D_{\mathrm{K}\mathrm{L}}(q({\mathbf{x}}_t|{\mathbf{x}}_{t+1},{\mathbf{x}}_0)||p_{\theta }({\mathbf{x}}_t|{\mathbf{x}}_{t+1}))\\ L_0& =-\log p_{\theta }({\mathbf{x}}_0|{\mathbf{x}}_1)\end{array}$$$L_{\mathrm{V}\mathrm{L}\mathrm{B}}$ Each of the KL term （ except $L_0$） Measure the distance between two Gaussian distributions , Therefore, they can be calculated with closed form solutions .$L_T$ Is constant , It can be ignored in the training process , And the reason is that $q$ There are no parameters to learn and ${\mathbf{x}}_T$ It's Gaussian noise ,$L_0$ It can be downloaded from $\mathcal{N}({\mathbf{x}}_0,{\mathbf{\mu }}_{\theta }({\mathbf{x}}_1,1),{\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_1,1)$ It's derived from .

5 Parameterization of training loss

  When it is necessary to learn a neural network to approximate the conditional probability distribution in the reverse diffusion process $p_{\theta }({\mathbf{x}}_{t-1}|{\mathbf{x}}_t)=\mathcal{N}({\mathbf{x}}_{t-1};{\mathbf{\mu }}_{\theta }({\mathbf{x}}_t,t),{\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t))$ when , I want to train ${\mathbf{\mu }}_{\theta }$ forecast ${\tilde{\mathbf{\mu }}}_t=\frac{1}{\sqrt{{\alpha }_t}}\left(\mathbf{x}-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathbf{z}}_t\right)$. because ${\mathbf{x}}_t$ It can be used as input during training , The Gaussian noise term can be re parameterized , So that it changes from time step $t$ The input of ${\mathbf{x}}_t$ Medium forecast ${\mathbf{z}}_t$：
$$\begin{array}{rl}{\mathbf{\mu }}_{\theta }({\mathbf{x}}_t,t)& =\frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathbf{z}}_{\theta }({\mathbf{x}}_t,t)\right)\\ {\mathbf{x}}_{t-1}& =\mathcal{N}\left({\mathbf{x}}_{t-1};\frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathbf{z}}_{\theta }({\mathbf{x}}_t,t)\right),{\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t)\right)\end{array}$$ Loss item $L_t$ Is parameterized in order to minimize from $\tilde{\mathbf{\mu }}$ The difference of $$\begin{array}{rl}{L}_t& ={\mathbb{E}}_{{\mathbf{x}}_0,\mathbf{z}}\left[\frac{1}{2\Vert {\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t){\Vert }_2^2}\Vert {\tilde{\mathbf{\mu }}}_t({\mathbf{x}}_t,{\mathbf{x}}_0)-{\mathbf{\mu }}_{\theta }({\mathbf{x}}_t,t){\Vert }^2\right]\\ & ={\mathbb{E}}_{{\mathbf{x}}_0,\mathbf{z}}\left[\frac{1}{2\Vert {\mathbf{\Sigma }}_{\theta }{\Vert }_2^2}\Vert \frac{1}{{\sqrt{\alpha }}_t}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-\bar{\alpha }}}\mathbf{z}\right)-\frac{1}{{\sqrt{\alpha }}_t}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-\bar{\alpha }}}{\mathbf{z}}_{\theta }({\mathbf{x}}_t,t)\right)\Vert \right]\\ & ={\mathbb{E}}_{{\mathbf{x}}_0,\mathbf{z}}\left[\frac{{\beta }_t^2}{2{\alpha }_t(1-{\bar{\alpha }}_t)\Vert {\mathbf{\Sigma }}_{\theta }{\Vert }_2^2}\Vert {\mathbf{z}}_t-{\mathbf{z}}_{\theta }({\mathbf{x}}_t,t){\Vert }^2\right]\\ & ={\mathbb{E}}_{{\mathbf{x}}_0,\mathbf{z}}\left[\frac{{\beta }_t^2}{2{\alpha }_t(1-{\bar{\alpha }}_t)\Vert {\mathbf{\Sigma }}_{\theta }{\Vert }_2^2}\Vert {\mathbf{z}}_t-{\mathbf{z}}_{\theta }(\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}{\mathbf{z}}_t,t){\Vert }^2\right]\end{array}$$ Based on experience Ho The experience of others , It is found that under the simplification goal of ignoring the weighted term , The effect of training diffusion model is better ：$$L_t^{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}={\mathbb{E}}_{{\mathbf{x}}_0,{\mathbf{z}}_t}\left[\Vert {\mathbf{z}}_t-{\mathbf{z}}_{\theta }(\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}{\mathbf{z}}_t,t){\Vert }^2\right]$$ So the final simplified objective function is ：$$L_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}=L^{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}+C$$ among $C$ Whether it depends on $\theta$ The constant .

6 Noise rating condition network （NCSN）

 Song and Ermon Et al. Proposed a fraction based generation modeling method , The sample is passed Langevin Dynamics is generated using the gradient of data distribution estimated by fractional matching . Each sample $\mathbf{x}$ The density probability score of is defined as its gradient ${\nabla }_{\mathbf{x}}\log p(\mathbf{x})$. Train a score network $s_{\theta }:{\mathbb{R}}^D\to {\mathbb{R}}^D$ To estimate it . In order to use high-dimensional data in deep learning settings to make it scalable , Some studies suggest using denoising score matching （ Add pre specified small noise to the data ） Or slice score matching .Langevin Dynamics can sample data points from the probability density distribution using only the fraction in the iteration process ${\nabla }_{\mathbf{x}}\log p(\mathbf{x})$. However , According to the manifold hypothesis , Most of the data is expected to be concentrated in low dimensional manifolds , Even the observed data may seem to be of arbitrary high dimensions . Because data points cannot cover the whole space ${\mathbb{R}}^D$, Therefore, it has a negative impact on score estimation . In areas with low data density , The score estimation is not reliable . Add a small Gaussian noise to make the disturbed data distribution cover the whole space , The training of score evaluation network becomes more stable . Song and Ermon Et al. Improved it by disturbing data with different levels of noise , A noise condition scoring network is trained to jointly estimate the scores of all disturbed data under different noise levels .

7 ${\beta }_t$ and ${\mathbf{\Sigma }}_{\theta }$ Parameterization of

  A parameterized ${\beta }_t$ In the process of ,Ho Et al. Set the forward variance as a series of linearly increasing constants , from ${\beta }_1=10^{-4}$ To ${\beta }_T=0.02$. And $[-1,1]$ Between normalized image pixel values , They are relatively small . Under this setting, the diffusion model in the experiment generates high-quality samples , But it still can't be competitive like other generation models .Nichol and Dhariwal Et al. Proposed several improved techniques to help the diffusion model achieve lower NLL. One of the improvements is the use of cosine based variance plans . The choice of scheduling function can be arbitrary , As long as it provides a near linear descent and surround in the middle of the training process $t=0$ and $t=T$ Subtle changes in $${\beta }_t=\mathrm{c}\mathrm{l}\mathrm{i}\mathrm{p}(1-\frac{{\bar{\alpha }}_t}{{\alpha }_{t-1}},0.999){\bar{\alpha }}_t=\frac{f(t)}{f(0)}\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}f(t)=\cos (\frac{t/T+s}{1+s}\cdot \frac{\pi }{2})$$ Among them $t=0$ Small offset $s$ To prevent ${\beta }_t$ Too small when approaching .
  A parameterized ${\mathbf{\Sigma }}_{\theta }$ In the process of ,Ho Others choose to fix ${\beta }_t$ For constant , Instead of making them learnable and settable ${\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t)={\sigma }_t^2\mathbf{I}$, among ${\sigma }_t$ You can't learn . The experiment found that learning diagonal variance ${\mathbf{\Sigma }}_{\theta }$ It will lead to unstable training and decreased sample quality .Nichol and Dhariwal Others proposed to learn ${\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t)$ As $\beta$ and ${\tilde{\beta }}_t$ Interpolation between , Predict the mixing vector through the model $\mathbf{v}$, Then there are ：$${\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t)=\exp (\mathbf{v}\log {\beta }_t+(1-\mathbf{v})\log {\tilde{\beta }}_t)$$ Simple goals $L_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}$ It doesn't depend on ${\mathbf{\Sigma }}_{\theta }$. To increase dependency , They built a hybrid goal $L_{\mathrm{h}\mathrm{y}\mathrm{b}\mathrm{r}\mathrm{i}\mathrm{d}}=L_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}+\lambda L_{\mathrm{V}\mathrm{L}\mathrm{B}}$, among $\lambda =0.001$ It's small and stops at ${\mathbf{\mu }}_{\theta }$ Gradient of , In order to $L_{\mathrm{V}\mathrm{L}\mathrm{B}}$ Guidance only ${\mathbf{\Sigma }}_{\theta }$ Learning from . Can be observed , Due to gradient noise , Optimize $L_{\mathrm{V}\mathrm{L}\mathrm{B}}$ It's very difficult , Therefore, they suggest using a time averaged smoothing version of importance sampling .

8 Accelerated diffusion model sampling

  By following the Markov chain of the reverse diffusion process from DDPM Generating samples is very slow , It may take one or thousands of steps . from DDPM In the sample $50000$ Size is $32\times 32$ The image of needs about $20$ Hours , But from Nvidia 2080 Ti GPU Upper GAN In less than a minute . A simple method is to run a step sampling plan , Sample and update each step , To reduce the intermediate sampling process . For the other way , Need to rewrite $q_{\sigma }({\mathbf{x}}_t|{\mathbf{x}}_t,{\mathbf{x}}_0)$ To pass the required standard deviation ${\sigma }_t$ Parameterize ：$$\begin{array}{rl}{\mathbf{x}}_{t-1}& =\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_{t-1}}{\mathbf{z}}_{t-1}\\ & =\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_{t-1}-{\sigma }_t^2{\mathbf{z}}_t}+{\sigma }_t\mathbf{z}\\ & =\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_{t-1}-{\sigma }_t^2}\frac{{\mathbf{x}}_t-\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0}{\sqrt{1-{\bar{\alpha }}_t}}+{\sigma }_t\mathbf{z}\\ q_{\sigma }& ({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)=\mathcal{N}\left({\mathbf{x}}_{t-1};\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_{t-1}-{\sigma }_t^2}\frac{{\mathbf{x}}_t-\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0}{1-{\bar{\alpha }}_t},{\sigma }_t^2\mathbf{I}\right)\end{array}$$ because $q({\mathbf{x}}_{t-1}|{\mathbf{x}}_t,{\mathbf{x}}_0)=\mathcal{N}({\mathbf{x}}_{t-1};\tilde{\mathbf{\mu }}({\mathbf{x}}_t,{\mathbf{x}}_0,{\tilde{\beta }}_t\mathbf{I}))$, So there is $${\tilde{\beta }}_t={\sigma }_t^2=\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}\cdot {\beta }_t$$ Make ${\sigma }_t^2=\eta \cdot {\tilde{\beta }}_t$, Then it can be adjusted to a super parameter $\eta \in {\mathbb{R}}^{+}$ To control the randomness of sampling .$\eta =0$ The special situation of makes the sampling process deterministic , Such a model is named denoising diffusion implicit model （DDIM）.DDIM Have the same marginal noise distribution , But the noise is definitely mapped back to the original data sample . In the process of generation , Only for a subset of diffusion steps $S$ Sampling is $\{{\tau }_1,\cdots ,{\tau }_S\}$, The reasoning process becomes ：$$q_{\sigma ,\tau }({\mathbf{x}}_{{\tau }_{i-1}}|{\mathbf{x}}_{{\tau }_t},{\mathbf{x}}_0)=\mathcal{N}({\mathbf{x}}_{{\tau }_{i-1}};\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_{t-1}-{\sigma }_t^2}\frac{{\mathbf{x}}_{{\tau }_i}-\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0}{\sqrt{1-{\bar{\alpha }}_t}},{\sigma }_t^2\mathbf{I})$$ Can be observed DDIM In the case of a small number of samples, the best quality samples can be produced , and DDPM The performance is much worse with a small number of samples . Use DDIM The diffusion model can be trained to any number of forward steps , However, only a subset of steps in the generation process can be sampled . In conclusion , And DDPM comparison ,DDIM Advantages as follows ：

• Use fewer steps to generate higher quality samples .
• Because the generation process is deterministic , Therefore has “ Uniformity ” attribute , This means that multiple samples with the same implicit variable as the condition should have similar high-level characteristics .
• Because of consistency ,DDIM Semantically meaningful interpolation can be carried out in implicit variables .

9 Conditional generation

  stay ImageNet When training and generating models on data , Samples that are conditional on class labels are usually generated . In order to explicitly incorporate category information into the diffusion process ,Dhariwal and Nichol For noisy images ${\mathbf{x}}_t$ Trained a classifier $f_{\varphi }(y|{\mathbf{x}}_t,t)$, And use gradients ${\nabla }_{\mathbf{x}}\log f_{\varphi }(y|{\mathbf{x}}_t,t)$ To guide the diffusion sampling process towards the target category label $y$. Ablation diffusion model (ADM) And a model with additional classifier guidance (ADM-G) Can get the best generation model than the current （BigGAN） Better results . Besides ,Dhariwal and Nichol And so on UNet Make some changes to the architecture , It shows that... With diffusion model GAN Better performance . Model architecture modifications include greater model depth / Width 、 More attention head 、 Multiresolution attention 、 Used for the / Down sampled BigGAN Residual block 、 Residual connection rescaling and adaptive group normalization (AdaGN).