Low-dimensional adaptation of diffusion models: Convergence in total variation
It offers rigorous theoretical evidence for the adaptivity of diffusion samplers to low-dimensional structure, improving over prior DDPM theory, which is incremental but important for researchers in generative modeling.
This paper investigates how diffusion generative models accelerate sampling by leveraging low-dimensional structure, proving that DDIM and DDPM samplers achieve iteration complexities of order k/ε for target distributions without smoothness or log-concavity assumptions, and providing a lower bound suggesting the necessity of certain coefficients for adaptation.
This paper investigates how diffusion generative models leverage (unknown) low-dimensional structure to accelerate sampling. Focusing on two mainstream samplers -- the denoising diffusion implicit model (DDIM) and the denoising diffusion probabilistic model (DDPM) -- and assuming accurate score estimates, we prove that their iteration complexities are no greater than the order of $k/\varepsilon$ (up to some log factor), where $\varepsilon$ is the precision in total variation distance and $k$ is some intrinsic dimension of the target distribution. Our results are applicable to a broad family of target distributions without requiring smoothness or log-concavity assumptions. Further, we develop a lower bound that suggests the (near) necessity of the coefficients introduced by Ho et al.(2020) and Song et al.(2020) in facilitating low-dimensional adaptation. Our findings provide the first rigorous evidence for the adaptivity of the DDIM-type samplers to unknown low-dimensional structure, and improve over the state-of-the-art DDPM theory regarding total variation convergence.