Diffusion Model

Destroying an image is easy: add a bit of random noise, then a bit more, and after enough steps you have pure static. That is the forward process, and it needs no learning.

A diffusion model learns to run the film backward. Training is simple:

1. Take a real image and a random step number $t$.
2. Add the amount of noise that belongs to step $t$.
3. Ask the network: what noise was added? Compare with the truth and update.

After training on millions of images, the network is an expert denoiser at every noise level. To generate, start from pure random noise and repeatedly subtract the predicted noise, a little at a time. Each step makes the image slightly cleaner, and after dozens of steps a new image emerges that never existed in the training set.

Text conditioning: feed a text description into the denoiser (through attention layers), and it learns to denoise toward images that match the text. Classifier-free guidance exaggerates the difference between “denoise with the prompt” and “denoise without it”, trading variety for prompt-faithfulness.

Latent diffusion: running hundreds of steps on full-size images is expensive, so models like Stable Diffusion first compress the image with an autoencoder (see VAE) and do the diffusion in that small latent space.

The math

The forward process has a closed form, with noise schedule $\bar\alpha_t$:

$$x_t = \sqrt{\bar\alpha_t}\, x_0 + \sqrt{1 - \bar\alpha_t}\, \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I)$$

The network $\epsilon_\theta$ is trained with the simple loss

$$\mathcal{L} = \mathbb{E}_{x_0, t, \epsilon}\big[\lVert \epsilon - \epsilon_\theta(x_t, t) \rVert^2\big]$$