Tutorial: Deriving the Standard Variational Autoencoder (VAE) Loss Function
This is an incremental tutorial for learners in Bayesian machine learning, providing a step-by-step derivation without introducing new methods.
The paper tackles the problem of deriving the loss function for Variational Autoencoders (VAEs) by explaining the variational inference approach used to approximate intractable posterior distributions, resulting in a detailed tutorial that includes closed-form solutions under Gaussian assumptions.
In Bayesian machine learning, the posterior distribution is typically computationally intractable, hence variational inference is often required. In this approach, an evidence lower bound on the log likelihood of data is maximized during training. Variational Autoencoders (VAE) are one important example where variational inference is utilized. In this tutorial, we derive the variational lower bound loss function of the standard variational autoencoder. We do so in the instance of a gaussian latent prior and gaussian approximate posterior, under which assumptions the Kullback-Leibler term in the variational lower bound has a closed form solution. We derive essentially everything we use along the way; everything from Bayes' theorem to the Kullback-Leibler divergence.