Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers
This provides theoretical guarantees for deterministic diffusion samplers, addressing a gap in the literature for researchers and practitioners in generative modeling.
The paper tackles the problem of analyzing deterministic samplers in diffusion generative modeling, which lacked non-asymptotic convergence bounds, by introducing a new operational interpretation that decomposes the probability flow ODE into restoration and degradation steps, enabling the first polynomial convergence bounds under mild conditions.
We develop a framework for non-asymptotic analysis of deterministic samplers used for diffusion generative modeling. Several recent works have analyzed stochastic samplers using tools like Girsanov's theorem and a chain rule variant of the interpolation argument. Unfortunately, these techniques give vacuous bounds when applied to deterministic samplers. We give a new operational interpretation for deterministic sampling by showing that one step along the probability flow ODE can be expressed as two steps: 1) a restoration step that runs gradient ascent on the conditional log-likelihood at some infinitesimally previous time, and 2) a degradation step that runs the forward process using noise pointing back towards the current iterate. This perspective allows us to extend denoising diffusion implicit models to general, non-linear forward processes. We then develop the first polynomial convergence bounds for these samplers under mild conditions on the data distribution.