Reducing the Amortization Gap in Variational Autoencoders: A Bayesian Random Function Approach
This work addresses the problem of improving the accuracy of posterior approximation in Variational Autoencoders for researchers and practitioners using generative models, offering a faster alternative to semi-amortized methods.
This paper tackles the problem of degraded accuracy in posterior approximation in VAEs due to amortized inference. They propose modeling the mean and variance functions of the variational posterior as random Gaussian processes, which allows for quantifying uncertainty in posterior approximation. The proposed method achieves higher test data likelihood than state-of-the-art approaches on several benchmark datasets.
Variational autoencoder (VAE) is a very successful generative model whose key element is the so called amortized inference network, which can perform test time inference using a single feed forward pass. Unfortunately, this comes at the cost of degraded accuracy in posterior approximation, often underperforming the instance-wise variational optimization. Although the latest semi-amortized approaches mitigate the issue by performing a few variational optimization updates starting from the VAE's amortized inference output, they inherently suffer from computational overhead for inference at test time. In this paper, we address the problem in a completely different way by considering a random inference model, where we model the mean and variance functions of the variational posterior as random Gaussian processes (GP). The motivation is that the deviation of the VAE's amortized posterior distribution from the true posterior can be regarded as random noise, which allows us to take into account the uncertainty in posterior approximation in a principled manner. In particular, our model can quantify the difficulty in posterior approximation by a Gaussian variational density. Inference in our GP model is done by a single feed forward pass through the network, significantly faster than semi-amortized methods. We show that our approach attains higher test data likelihood than the state-of-the-arts on several benchmark datasets.