On the Trajectory Regularity of ODE-based Diffusion Sampling
This work addresses the computational bottleneck in diffusion models for image generation, offering an incremental improvement with minimal overhead.
The paper tackles the problem of improving sampling efficiency in ODE-based diffusion models by analyzing trajectory regularity and optimizing time schedules, resulting in superior image generation performance with only 5-10 function evaluations.
Diffusion-based generative models use stochastic differential equations (SDEs) and their equivalent ordinary differential equations (ODEs) to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models. We characterize an implicit denoising trajectory and discuss its vital role in forming the coupled sampling trajectory with a strong shape regularity, regardless of the generated content. We also describe a dynamic programming-based scheme to make the time schedule in sampling better fit the underlying trajectory structure. This simple strategy requires minimal modification to any given ODE-based numerical solvers and incurs negligible computational cost, while delivering superior performance in image generation, especially in $5\sim 10$ function evaluations.