Discrete vs. Continuous Trade-offs for Generative Models
Provides theoretical analysis of diffusion models for researchers in generative modeling.
This paper analyzes theoretical performance bounds for denoising diffusion probabilistic models (DDPMs) and score-based generative models, demonstrating how score estimation errors propagate through reverse diffusion processes and bounding total variation distance using mathematical tools like Girsanov transformations and information inequalities.
This work explores the theoretical and practical foundations of denoising diffusion probabilistic models (DDPMs) and score-based generative models, which leverage stochastic processes and Brownian motion to model complex data distributions. These models employ forward and reverse diffusion processes defined through stochastic differential equations (SDEs) to iteratively add and remove noise, enabling high-quality data generation. By analyzing the performance bounds of these models, we demonstrate how score estimation errors propagate through the reverse process and bound the total variation distance using discrete Girsanov transformations, Pinsker's inequality, and the data processing inequality (DPI) for an information theoretic lens.