Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory
This work addresses a foundational gap in theory for researchers and practitioners using conditional diffusion models in fields like image synthesis and computational biology, though it is incremental as it builds on existing empirical successes.
The paper tackles the lack of theoretical understanding for conditional diffusion models by developing a sharp statistical theory that provides a sample complexity bound matching the minimax lower bound, adapting to data distribution smoothness.
Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. In these applications, conditional diffusion models incorporate various conditional information, such as prompt input, to guide the sample generation towards desired properties. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges this gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models. Our analysis yields a sample complexity bound that adapts to the smoothness of the data distribution and matches the minimax lower bound. The key to our theoretical development lies in an approximation result for the conditional score function, which relies on a novel diffused Taylor approximation technique. Moreover, we demonstrate the utility of our statistical theory in elucidating the performance of conditional diffusion models across diverse applications, including model-based transition kernel estimation in reinforcement learning, solving inverse problems, and reward conditioned sample generation.