A Frequentist Statistical Introduction to Variational Inference, Autoencoders, and Diffusion Models
This work addresses a barrier for statisticians by contextualizing VI, VAEs, and DDMs from a Frequentist perspective, making it incremental as it synthesizes existing knowledge without new empirical results.
The paper tackles the pedagogical gap in understanding Variational Inference (VI) in generative models like VAEs and DDMs by providing a Frequentist introduction, showing how VI arises from the EM algorithm and bridges classical statistics with modern AI.
While Variational Inference (VI) is central to modern generative models like Variational Autoencoders (VAEs) and Denoising Diffusion Models (DDMs), its pedagogical treatment is split across disciplines. In statistics, VI is typically framed as a Bayesian method for posterior approximation. In machine learning, however, VAEs and DDMs are developed from a Frequentist viewpoint, where VI is used to approximate a maximum likelihood estimator. This creates a barrier for statisticians, as the principles behind VAEs and DDMs are hard to contextualize without a corresponding Frequentist introduction to VI. This paper provides that introduction: we explain the theory for VI, VAEs, and DDMs from a purely Frequentist perspective, starting with the classical Expectation-Maximization (EM) algorithm. We show how VI arises as a scalable solution for intractable E-steps and how VAEs and DDMs are natural, deep-learning-based extensions of this framework, thereby bridging the gap between classical statistical inference and modern generative AI.