On the Mathematics of Diffusion Models
This work clarifies the foundational mathematics of diffusion models for researchers in machine learning, though it is incremental as it re-derives known results without new empirical gains.
The paper provides direct mathematical derivations of the differential equations and likelihood formulas for diffusion models, assuming only knowledge of Gaussian distributions, resulting in explicit expressions for forward/backward stochastic differential equations (SDEs), non-variational integral likelihood formulas, and reverse diffusion ordinary differential equations (ODEs).
This paper gives direct derivations of the differential equations and likelihood formulas of diffusion models assuming only knowledge of Gaussian distributions. A VAE analysis derives both forward and backward stochastic differential equations (SDEs) as well as non-variational integral expressions for likelihood formulas. A score-matching analysis derives the reverse diffusion ordinary differential equation (ODE) and a family of reverse-diffusion SDEs parameterized by noise level. The paper presents the mathematics directly with attributions saved for a final section.