A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models
This provides a theoretical foundation for diffusion model samplers, addressing a key bottleneck in generative modeling for researchers and practitioners, though it is incremental as it builds on prior convergence theories.
The paper tackles the problem of establishing non-asymptotic convergence bounds for the probability flow ODE sampler in diffusion models, proving that approximately d/ε iterations are sufficient to approximate a target distribution in ℝᵈ within ε total-variation distance, which is the first result showing nearly linear dimension-dependency.
Diffusion models, which convert noise into new data instances by learning to reverse a diffusion process, have become a cornerstone in contemporary generative modeling. In this work, we develop non-asymptotic convergence theory for a popular diffusion-based sampler (i.e., the probability flow ODE sampler) in discrete time, assuming access to $\ell_2$-accurate estimates of the (Stein) score functions. For distributions in $\mathbb{R}^d$, we prove that $d/\varepsilon$ iterations -- modulo some logarithmic and lower-order terms -- are sufficient to approximate the target distribution to within $\varepsilon$ total-variation distance. This is the first result establishing nearly linear dimension-dependency (in $d$) for the probability flow ODE sampler. Imposing only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), our results also characterize how $\ell_2$ score estimation errors affect the quality of the data generation processes. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach without the need of resorting to SDE and ODE toolboxes.