Optimization-Free Diffusion Model -- A Perturbation Theory Approach
This addresses the computational inefficiency for researchers and practitioners using diffusion models, though it appears incremental as it builds on existing diffusion frameworks.
The paper tackles the computational burden of diffusion models by proposing an optimization-free and forward SDE-free method that reformulates score estimation as a linear system using eigenbasis expansion, achieving effectiveness on high-dimensional Boltzmann distributions and real-world datasets.
Diffusion models have emerged as a powerful framework in generative modeling, typically relying on optimizing neural networks to estimate the score function via forward SDE simulations. In this work, we propose an alternative method that is both optimization-free and forward SDE-free. By expanding the score function in a sparse set of eigenbasis of the backward Kolmogorov operator associated with the diffusion process, we reformulate score estimation as the solution to a linear system, avoiding iterative optimization and time-dependent sample generation. We analyze the approximation error using perturbation theory and demonstrate the effectiveness of our method on high-dimensional Boltzmann distributions and real-world datasets.