Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data
This addresses the theoretical gap in diffusion models for researchers, showing they can circumvent the curse of dimensionality in low-dimensional data settings.
This paper provides theoretical guarantees for diffusion models when data lies on an unknown low-dimensional subspace, showing that with proper neural network architecture, the score function can be accurately approximated and estimated, leading to generated distributions that converge to the data distribution with rates depending on subspace dimension rather than ambient dimension.
Diffusion models achieve state-of-the-art performance in various generation tasks. However, their theoretical foundations fall far behind. This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. Our result provides sample complexity bounds for distribution estimation using diffusion models. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. Furthermore, the generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution. The convergence rate depends on the subspace dimension, indicating that diffusion models can circumvent the curse of data ambient dimensionality.