Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach
This work addresses the theoretical gap in accelerated diffusion models for broader target distributions, which is incremental but important for practitioners needing efficient sampling from diverse distributions.
The paper tackles the limitation of accelerated diffusion models to restrictive target distribution classes by developing a new accelerated stochastic DDPM sampler that achieves accelerated performance for three broader distribution classes: those with smoothness only at the target density, those with finite second moments, and Gaussian mixtures, with improved dimension dependency for bounded-support cases.
Accelerated diffusion models hold the potential to significantly enhance the efficiency of standard diffusion processes. Theoretically, these models have been shown to achieve faster convergence rates than the standard $\mathcal O(1/ε^2)$ rate of vanilla diffusion models, where $ε$ denotes the target accuracy. However, current theoretical studies have established the acceleration advantage only for restrictive target distribution classes, such as those with smoothness conditions imposed along the entire sampling path or with bounded support. In this work, we significantly broaden the target distribution classes with a new accelerated stochastic DDPM sampler. In particular, we show that it achieves accelerated performance for three broad distribution classes not considered before. Our first class relies on the smoothness condition posed only to the target density $q_0$, which is far more relaxed than the existing smoothness conditions posed to all $q_t$ along the entire sampling path. Our second class requires only a finite second moment condition, allowing for a much wider class of target distributions than the existing finite-support condition. Our third class is Gaussian mixture, for which our result establishes the first acceleration guarantee. Moreover, among accelerated DDPM type samplers, our results specialized for bounded-support distributions show an improved dependency on the data dimension $d$. Our analysis introduces a novel technique for establishing performance guarantees via constructing a tilting factor representation of the convergence error and utilizing Tweedie's formula to handle Taylor expansion terms. This new analytical framework may be of independent interest.