Linear Convergence of Diffusion Models Under the Manifold Hypothesis
This provides a theoretical guarantee for faster sampling in generative models, which is incremental but important for applications relying on efficient high-dimensional data generation.
The paper tackles the problem of slow convergence in diffusion models for high-dimensional data by proving that, under the manifold hypothesis, the number of steps required for convergence in KL divergence is linear in the intrinsic dimension d, and shows this linear dependency is sharp.
Score-matching generative models have proven successful at sampling from complex high-dimensional data distributions. In many applications, this distribution is believed to concentrate on a much lower $d$-dimensional manifold embedded into $D$-dimensional space; this is known as the manifold hypothesis. The current best-known convergence guarantees are either linear in $D$ or polynomial (superlinear) in $d$. The latter exploits a novel integration scheme for the backward SDE. We take the best of both worlds and show that the number of steps diffusion models require in order to converge in Kullback-Leibler~(KL) divergence is linear (up to logarithmic terms) in the intrinsic dimension $d$. Moreover, we show that this linear dependency is sharp.