Shallow diffusion networks provably learn hidden low-dimensional structure
This work addresses the theoretical gap in understanding why diffusion models succeed empirically in high-dimensional tasks like image generation, offering a foundational insight for researchers in machine learning theory.
The paper tackles the curse of dimensionality in diffusion-based generative models by proving that shallow diffusion networks can adapt to low-dimensional structure in distributions, avoiding this curse, and provides an end-to-end sample complexity bound for learning to sample from such structured distributions.
Diffusion-based generative models provide a powerful framework for learning to sample from a complex target distribution. The remarkable empirical success of these models applied to high-dimensional signals, including images and video, stands in stark contrast to classical results highlighting the curse of dimensionality for distribution recovery. In this work, we take a step towards understanding this gap through a careful analysis of learning diffusion models over the Barron space of single layer neural networks. In particular, we show that these shallow models provably adapt to simple forms of low dimensional structure, thereby avoiding the curse of dimensionality. We combine our results with recent analyses of sampling with diffusion models to provide an end-to-end sample complexity bound for learning to sample from structured distributions. Importantly, our results do not require specialized architectures tailored to particular latent structures, and instead rely on the low-index structure of the Barron space to adapt to the underlying distribution.