DDPM模型——公式推导

论文传送门：Denoising Diffusion Probabilistic Models
代码实现：DDPM模型——pytorch实现
推荐视频：54、Probabilistic Diffusion Model概率扩散模型理论与完整PyTorch代码详细解读

需要的数学基础：

联合概率(Joint probability)：
$$P(A,B,C)=P(C\mid B,A)P(B,A)=P(C\mid B,A)P(B\mid A)P(A)$$
条件概率(Conditional probability)：
$$P(B,C\mid A)=P(B\mid A)P(C\mid A,B)$$
马尔可夫链(Markov Chain)：
$$p\left(X_{t+1}\mid X_t,\dots ,X_1\right)=p\left(X_{t+1}\mid X_t\right)$$
贝叶斯公式(Bayes Rule)：
$$P\left(A_i\mid B\right)=\frac{P\left(B\mid A_i\right)P\left(A_i\right)}{{\sum }_{j}P\left(B\mid A_j\right)P\left(A_j\right)}$$
正态分布(Normal distribution)$X\sim N\left(\mu ,{\sigma }^2\right)$的概率密度函数：
$$f(x)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$$
两个正态分布$X\sim N\left({\mu }_X,{\sigma }_X^2\right)$和$Y\sim N\left({\mu }_Y,{\sigma }_Y^2\right)$的叠加：
$$U=X+Y\sim N\left({\mu }_X+{\mu }_Y,{\sigma }_X^2+{\sigma }_Y^2\right)$$
两个正态分布$p,q$的KL散度(Kullback-Leibler divergence)：
$$KL(p,q)=\log \frac{{\sigma }_q}{{\sigma }_p}+\frac{{\sigma }_p^2+{\left({\mu }_p-{\mu }_q\right)}^2}{2{\sigma }_q^2}-\frac{1}{2}$$
重参数技巧(Reparameterrization)：
$$若X\sim N\left(\mu ,{\sigma }^2\right),Y=\frac{X-\mu }{\sigma }\sim N(0,1)$$
从正态分布$X$中采样$z$，等价于从标准正态分布$Y$中采样$z^{\prime }$，$z=\mu +\sigma \times z^{\prime }$
一元二次式的配方：
$$a{x}^2+bx=a{\left(x+\frac{b}{2a}\right)}^2+C$$

概念：

$t$：时刻(加噪次数)
$T$：总时长(总加噪次数)
$\mathbf{x}$：图像
${\mathbf{x}}_0$：初始时刻图像
${\mathbf{x}}_t$：$t$时刻图像
${\mathbf{x}}_T$：终止时刻图像
$x_0$ ~ $q(x_0)$，$q(x_0)$：真实图像分布
$p_{\theta }(x_0):=\int p_{\theta }(x_{0:T})d{x}_{1:T}$，$p_{\theta }(x_0)$：生成图像分布
$\theta$：(网络)参数
${\beta }_t$：扩散过程t时刻加入噪声的方差
$\beta$：噪声方差序列，长度为T，在$(0,1)$区间内单调递增

Reverse process：

逆扩散过程的数学表达：
$$p_{\theta }\left({\mathbf{x}}_{0:T}\right):=p\left({\mathbf{x}}_T\right)\prod _{t=1}^T{p}_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)$$
全部时刻图像的联合概率分布$p_{\theta }\left({\mathbf{x}}_{0:T}\right)$，整个过程是马尔科夫链。其中，
$$p\left({\mathbf{x}}_T\right)=\mathcal{N}\left({\mathbf{x}}_T;0,\mathbf{I}\right)$$
$p\left({\mathbf{x}}_T\right)$是标准正态分布，${\mathbf{x}}_T$为采样值，与网络参数无关。
$t$时刻去噪的数学表达：
$$p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right):=\mathcal{N}\left({\mathbf{x}}_{t-1};{\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right),{\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)\right)$$
${\mathbf{x}}_{t-1}$服从均值为${\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right)$，方差为${\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)$的正态分布，作者在原文中将方差${\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)$设为${\sigma }_t^2={\tilde{\beta }}_t=\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}{\beta }_t$(经实验，${\sigma }_t^2={\beta }_t$和${\sigma }_t^2={\tilde{\beta }}_t$的结果相似)，与模型参数无关(${\tilde{\beta }}_t$在后续计算中会提到)。

Forward process：

扩散过程的数学表达：
$$q\left({\mathbf{x}}_{1:T}\mid {\mathbf{x}}_0\right):=\prod _{t=1}^{T}q\left({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1}\right)$$
给定初始图像${\mathbf{x}}_0$，全部时刻($t>0$)的联合概率分布，整个过程是马尔科夫链。
$t$时刻加噪的数学表达：
$$q\left({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1}\right):=\mathcal{N}\left({\mathbf{x}}_t;\sqrt{1-{\beta }_t}{\mathbf{x}}_{t-1},{\beta }_t\mathbf{I}\right)$$
${\mathbf{x}}_t$服从均值为$\sqrt{1-{\beta }_t}{\mathbf{x}}_{t-1}$，方差为${\beta }_t$的正态分布。
使用重参数技巧，任意时刻的图像${\mathbf{x}}_t$可以由初始时刻图像${\mathbf{x}}_0$和噪声方差序列$\beta$来确定，为简化表达，定义${\alpha }_t:=1-{\beta }_t$，${\bar{\alpha }}_t:={\prod }_{s=1}^t{\alpha }_s$，则：
x t = α t x t − 1 + 1 − α t ϵ t − 1 = α t ( α t − 1 x t − 2 + 1 − α t − 1 ϵ t − 2 ) + 1 − α t ϵ t − 1 = α t α t − 1 x t − 2 + α t 1 − α t − 1 ϵ t − 2 + 1 − α t ϵ t − 1 = α t α t − 1 x t − 2 + α t − α t α t − 1 + 1 − α t ϵ ‾ t − 2 = α t α t − 1 x t − 2 + 1 − α t α t − 1 ϵ ‾ t − 2 = … = α ˉ t x 0 + 1 − α ˉ t ϵ xt=√αtxt−1+√1−αtϵt−1=√αt(√αt−1xt−2+√1−αt−1ϵt−2)+√1−αtϵt−1=√αtαt−1xt−2+√αt√1−αt−1ϵt−2+√1−αtϵt−1=√αtαt−1xt−2+√αt−αtαt−1+1−αt¯ϵt−2=√αtαt−1xt−2+√1−αtαt−1¯ϵt−2=…=√ˉαtx0+√1−ˉαtϵ

xt​​=αt​ ​xt−1​+1−αt​ ​ϵt−1​=αt​ ​(αt−1​ ​xt−2​+1−αt−1​ ​ϵt−2​)+1−αt​ ​ϵt−1​=αt​αt−1​ ​xt−2​+αt​ ​1−αt−1​ ​ϵt−2​+1−αt​ ​ϵt−1​=αt​αt−1​ ​xt−2​+αt​−αt​αt−1​+1−αt​ ​ϵt−2​=αt​αt−1​ ​xt−2​+1−αt​αt−1​ ​ϵt−2​=…=αˉt​ ​x0​+1−αˉt​ ​ϵ​
可以将上式改写，使用${\mathbf{x}}_t$和$ϵ$来表达${\mathbf{x}}_0$：
$${\mathbf{x}}_0=\frac{1}{\sqrt{{\bar{\alpha }}_t}}\left({\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}ϵ\right)$$
(推导过程用到两个正态分布的叠加公式)
其中，${ϵ}_i$ ~ $\mathcal{N}\left(0,\mathbf{I}\right)$

Loss：

负对数似然的上界：
E q [ − log ⁡ p θ ( x 0 ) ] ≤ E q [ − log ⁡ p θ ( x 0 ) ] + D K L ( q ( x 1 : T ∣ x 0 ) ∥ p ( x 1 : T ∣ x 0 ) ) = E q [ − log ⁡ p θ ( x 0 ) ] + E q [ log ⁡ q ( x 1 : T ∣ x 0 ) p θ ( x 0 : T ) / p θ ( x 0 ) ] = E q [ − log ⁡ p θ ( x 0 ) ] + E q [ − log ⁡ p θ ( x 0 : T ) q ( x 1 : T ∣ x 0 ) + log ⁡ p θ ( x 0 ) ] = E q [ − log ⁡ p θ ( x 0 : T ) q ( x 1 : T ∣ x 0 ) ] Eq[−logpθ(x0)]≤Eq[−logpθ(x0)]+DKL(q(x1:T∣x0)‖p(x1:T∣x0))=Eq[−logpθ(x0)]+Eq[logq(x1:T∣x0)pθ(x0:T)/pθ(x0)]=Eq[−logpθ(x0)]+Eq[−logpθ(x0:T)q(x1:T∣x0)+logpθ(x0)]=Eq[−logpθ(x0:T)q(x1:T∣x0)]

Eq​[−logpθ​(x0​)]​≤Eq​[−logpθ​(x0​)]+DKL​(q(x1:T​∣x0​)∥p(x1:T​∣x0​))=Eq​[−logpθ​(x0​)]+Eq​[logpθ​(x0:T​)/pθ​(x0​)q(x1:T​∣x0​)​]=Eq​[−logpθ​(x0​)]+Eq​[−logq(x1:T​∣x0​)pθ​(x0:T​)​+logpθ​(x0​)]=Eq​[−logq(x1:T​∣x0​)pθ​(x0:T​)​]​
定义损失函数L：
$${\mathbb{E}}_q\left[-\log p_{\theta }\left({\mathbf{x}}_0\right)\right]\le {\mathbb{E}}_q\left[-\log \frac{p_{\theta }\left({\mathbf{x}}_{0:T}\right)}{q\left({\mathbf{x}}_{1:T}\mid {\mathbf{x}}_0\right)}\right]:=L$$
L的进一步推导：
$$\begin{array}{rl}L& ={\mathbb{E}}_q\left[-\log \frac{p_{\theta }\left({\mathbf{x}}_{0:T}\right)}{q\left({\mathbf{x}}_{1:T}\mid {\mathbf{x}}_0\right)}\right]\\ & ={\mathbb{E}}_q\left[-\log p\left({\mathbf{x}}_T\right)-\sum _{t\ge 1}\log \frac{p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)}{q\left({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1}\right)}\right]\\ & ={\mathbb{E}}_q\left[-\log p\left({\mathbf{x}}_T\right)-\sum _{t>1}\log \frac{p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)}{q\left({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1}\right)}-\log \frac{p_{\theta }\left({\mathbf{x}}_0\mid {\mathbf{x}}_1\right)}{q\left({\mathbf{x}}_1\mid {\mathbf{x}}_0\right)}\right]\\ & ={\mathbb{E}}_q\left[-\log p\left({\mathbf{x}}_T\right)-\sum _{t>1}\log \frac{p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)}{q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)}\cdot \frac{q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_0\right)}{q\left({\mathbf{x}}_t\mid {\mathbf{x}}_0\right)}-\log \frac{p_{\theta }\left({\mathbf{x}}_0\mid {\mathbf{x}}_1\right)}{q\left({\mathbf{x}}_1\mid {\mathbf{x}}_0\right)}\right]\\ & ={\mathbb{E}}_q\left[-\log \frac{p\left({\mathbf{x}}_T\right)}{q\left({\mathbf{x}}_T\mid {\mathbf{x}}_0\right)}-\sum _{t>1}\log \frac{p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)}{q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)}-\log p_{\theta }\left({\mathbf{x}}_0\mid {\mathbf{x}}_1\right)\right]\\ & ={\mathbb{E}}_q\left[D_{\mathrm{KL}}\left(q\left({\mathbf{x}}_T\mid {\mathbf{x}}_0\right)\Vert p\left({\mathbf{x}}_T\right)\right)+\sum _{t>1}{D}_{\mathrm{KL}}\left(q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)\Vert p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)\right)-\log p_{\theta }\left({\mathbf{x}}_0\mid {\mathbf{x}}_1\right)\right]\end{array}$$
用$L_T$,$L_{t-1}$,$L_0$三项来表示：
$$L={\mathbb{E}}_q[\underset{L_T}{\underbrace{D_{\mathrm{KL}}\left(q\left({\mathbf{x}}_T\mid {\mathbf{x}}_0\right)\Vert p\left({\mathbf{x}}_T\right)\right)}}+\sum _{t>1}\underset{L_{t-1}}{\underbrace{D_{\mathrm{KL}}\left(q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)\Vert p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)\right)}}\underset{L_0}{\underbrace{-\log p_{\theta }\left({\mathbf{x}}_0\mid {\mathbf{x}}_1\right)}}]$$
第一项$L_T$与网络参数$\theta$无关，可以忽略。
对第三项$L_0$进行分析：
$$\begin{array}{rl}{p}_{\theta }\left({\mathbf{x}}_0\mid {\mathbf{x}}_1\right)& =\prod _{i=1}^D{\int }_{{\delta }_{-}\left(x_0^i\right)}^{{\delta }_{+}\left(x_0^i\right)}\mathcal{N}\left(x;{\mu }_{\theta }^i\left({\mathbf{x}}_1,1\right),{\sigma }_1^2\right)dx\\ {\delta }_{+}(x)& =\{\begin{array}{ll}\infty & \text{ if }x=1\\ x+\frac{1}{255}& \text{ if }x<1\end{array}{\delta }_{-}(x)=\{\begin{array}{ll}-\infty & \text{ if }x=-1\\ x-\frac{1}{255}& \text{ if }x>-1\end{array}\end{array}$$
相当于从连续空间向离散空间的变化，即将连续高斯分布转化为离散高斯分布，以对应输入的图片数据。
对第二项$L_{t-1}$进行分析：
KL散度的第一项$q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)$，设
$$q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)=\mathcal{N}\left({\mathbf{x}}_{t-1};{\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right),{\tilde{\beta }}_t\mathbf{I}\right)$$
使用贝叶斯公式和配方，计算$q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)$：
$$\begin{array}{rl}q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0\right)& =q\left({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1},{\mathbf{x}}_0\right)\frac{q\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_0\right)}{q\left({\mathbf{x}}_t\mid {\mathbf{x}}_0\right)}\\ & \propto \exp \left(-\frac{1}{2}\left(\frac{{\left({\mathbf{x}}_t-\sqrt{{\alpha }_t}{\mathbf{x}}_{t-1}\right)}^2}{{\beta }_t}+\frac{{\left({\mathbf{x}}_{t-1}-\sqrt{{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\right)}^2}{1-{\bar{\alpha }}_{t-1}}-\frac{{\left({\mathbf{x}}_t-\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0\right)}^2}{1-{\bar{\alpha }}_t}\right)\right)\\ & =\exp \left(-\frac{1}{2}\left(\left(\frac{{\alpha }_t}{{\beta }_t}+\frac{1}{1-{\bar{\alpha }}_{t-1}}\right){\mathbf{x}}_{t-1}^2-\left(\frac{2\sqrt{{\alpha }_t}}{{\beta }_t}{\mathbf{x}}_t+\frac{2\sqrt{{\alpha }_{t-1}}}{1-{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\right){\mathbf{x}}_{t-1}+C\left({\mathbf{x}}_t,{\mathbf{x}}_0\right)\right)\right)\end{array}$$
得到方差${\tilde{\beta }}_t$：
$$\begin{array}{rl}{\tilde{\beta }}_t& =1/\left(\frac{{\alpha }_t}{{\beta }_t}+\frac{1}{1-{\bar{\alpha }}_{t-1}}\right)\\ & =\frac{1-{\bar{\alpha }}_{t-1}}{{\alpha }_t+{\beta }_t-{\bar{\alpha }}_t}\cdot {\beta }_t\\ & =\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}\cdot {\beta }_t\end{array}$$
得到均值${\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right)$：
$$\begin{array}{rl}{\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right)& =\left(\frac{\sqrt{{\alpha }_t}}{{\beta }_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}}{1-{\bar{\alpha }}_{t-1}}{\mathbf{x}}_0\right)/\left(\frac{{\alpha }_t}{{\beta }_t}+\frac{1}{1-{\bar{\alpha }}_{t-1}}\right)\\ & =\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{{\alpha }_t+{\beta }_t-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{{\alpha }_t+{\beta }_t-{\bar{\alpha }}_t}{\mathbf{x}}_0\\ & =\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}{\mathbf{x}}_0\end{array}$$
使用${\mathbf{x}}_t$和$ϵ$来表达${\mathbf{x}}_0$：
$$\begin{array}{rl}{\tilde{\mathbf{\mu }}}_t& =\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}\frac{1}{\sqrt{{\bar{\alpha }}_t}}\left({\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}ϵ\right)\\ & =\frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}ϵ\right)\end{array}$$
KL散度的第二项$p_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)$，在逆扩散过程中已经定义，作者将方差${\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)$设为${\sigma }_t^2={\tilde{\beta }}_t$。
$$\begin{array}{rl}{p}_{\theta }\left({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t\right)& :=\mathcal{N}\left({\mathbf{x}}_{t-1};{\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right),{\mathbf{\Sigma }}_{\theta }\left({\mathbf{x}}_t,t\right)\right)\\ & =\mathcal{N}\left({\mathbf{x}}_{t-1};{\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right),{\mathbf{\sigma }}_t^2\right)\end{array}$$
使用两个正态分布(方差相同)的KL散度计算公式，可以计算得到：
$$L_{t-1}={\mathbb{E}}_q\left[\frac{1}{2{\sigma }_t^2}{\Vert {\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t,{\mathbf{x}}_0\right)-{\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right)\Vert }^2\right]+C$$
使用${\mathbf{x}}_0$和$ϵ$来表达${\mathbf{x}}_t$，$ϵ$ ~ $\mathcal{N}\left(0,\mathbf{I}\right)$：
$${\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right)=\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ}$$
于是，$L_{t-1}-C$可以表示为：
$$\begin{array}{rl}{L}_{t-1}-C& ={\mathbb{E}}_{{\mathbf{x}}_0,\mathbf{ϵ}}\left[\frac{1}{2{\sigma }_t^2}{\Vert {\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right),\frac{1}{\sqrt{{\bar{\alpha }}_t}}\left({\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right)-\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ}\right)\right)-{\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right),t\right)\Vert }^2\right]\\ & ={\mathbb{E}}_{{\mathbf{x}}_0,\mathbf{ϵ}}\left[\frac{1}{2{\sigma }_t^2}{\Vert \frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right)-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}\mathbf{ϵ}\right)-{\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right),t\right)\Vert }^2\right]\end{array}$$
上式表明，在给定${\mathbf{x}}_t$的情况下，需要网络输出${\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right),t\right)$去预测$\frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t\left({\mathbf{x}}_0,\mathbf{ϵ}\right)-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}\mathbf{ϵ}\right)$。
但作者并没有这样搭建网络，而是选择对${\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right)$进行参数化处理：
$${\mathbf{\mu }}_{\theta }\left({\mathbf{x}}_t,t\right)={\tilde{\mathbf{\mu }}}_t\left({\mathbf{x}}_t,\frac{1}{\sqrt{{\bar{\alpha }}_t}}\left({\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}{\mathbf{ϵ}}_{\theta }\left({\mathbf{x}}_t\right)\right)\right)=\frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathbf{ϵ}}_{\theta }\left({\mathbf{x}}_t,t\right)\right)$$
可见，网络通过输入${\mathbf{x}}_t$和$t$，实际输出为${ϵ}_{\theta }$(即预测噪声)，而${\mathbf{x}}_t$又可以由${\mathbf{x}}_0$来表示，最终$L_{t-1}-C$可以表示为：
$${\mathbb{E}}_{{\mathbf{x}}_0,\mathbf{ϵ}}\left[\frac{{\beta }_t^2}{2{\sigma }_t^2{\alpha }_t\left(1-{\bar{\alpha }}_t\right)}{\Vert \mathbf{ϵ}-{\mathbf{ϵ}}_{\theta }\left(\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ},t\right)\Vert }^2\right]$$
最终，作者忽略上式的系数，得到简化的Loss形式：
$$L_{\text{simple }}(\theta ):={\mathbb{E}}_{t,{\mathbf{x}}_0,\mathbf{ϵ}}\left[{\Vert \mathbf{ϵ}-{\mathbf{ϵ}}_{\theta }\left(\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ},t\right)\Vert }^2\right]$$