Non-asymptotic convergence bound of conditional diffusion models
This provides theoretical foundations for conditional diffusion models, addressing a gap in theoretical research for machine learning practitioners, though it is incremental as it builds on existing diffusion model frameworks.
The paper tackles the lack of non-asymptotic convergence bounds for conditional diffusion models by proposing CARD, which integrates a pre-trained model into the diffusion framework to approximate the original conditional distribution, and derives upper error bounds using the second-order Wasserstein distance under Lipschitz assumptions.
Learning and generating various types of data based on conditional diffusion models has been a research hotspot in recent years. Although conditional diffusion models have made considerable progress in improving acceleration algorithms and enhancing generation quality, the lack of non-asymptotic properties has hindered theoretical research. To address this gap, we focus on a conditional diffusion model within the domains of classification and regression (CARD), which aims to learn the original distribution with given input x (denoted as Y|X). It innovatively integrates a pre-trained model f_φ(x) into the original diffusion model framework, allowing it to precisely capture the original conditional distribution given f (expressed as Y|f_φ(x)). Remarkably, when f_φ(x) performs satisfactorily, Y|f_φ(x) closely approximates Y|X. Theoretically, we deduce the stochastic differential equations of CARD and establish its generalized form predicated on the Fokker-Planck equation, thereby erecting a firm theoretical foundation for analysis. Mainly under the Lipschitz assumptions, we utilize the second-order Wasserstein distance to demonstrate the upper error bound between the original and the generated conditional distributions. Additionally, by appending assumptions such as light-tailedness to the original distribution, we derive the convergence upper bound between the true value analogous to the score function and the corresponding network-estimated value.