Generative Diffusion From An Action Principle
This provides a theoretical framework for diffusion models, which is incremental as it builds on existing methods without introducing new practical gains.
The authors tackled the problem of understanding generative diffusion models by deriving score matching from an action principle, showing that reverse diffusion can be cast as an optimal control problem and connecting different classes of diffusion models.
Generative diffusion models synthesize new samples by reversing a diffusive process that converts a given data set to generic noise. This is accomplished by training a neural network to match the gradient of the log of the probability distribution of a given data set, also called the score. By casting reverse diffusion as an optimal control problem, we show that score matching can be derived from an action principle, like the ones commonly used in physics. We use this insight to demonstrate the connection between different classes of diffusion models.