Towards Deeper Understanding of Variational Autoencoding Models
This work addresses fundamental limitations in variational autoencoders for researchers in generative modeling, though it appears incremental as it builds on existing VAE frameworks.
The authors tackled the problem of blurry samples and uninformative latent features in variational autoencoders by proposing a new family of optimization criteria that generalize the standard evidence lower bound, leading to a sequential VAE model that generates sharp samples on the LSUN image dataset using pixel-wise reconstruction loss.
We propose a new family of optimization criteria for variational auto-encoding models, generalizing the standard evidence lower bound. We provide conditions under which they recover the data distribution and learn latent features, and formally show that common issues such as blurry samples and uninformative latent features arise when these conditions are not met. Based on these new insights, we propose a new sequential VAE model that can generate sharp samples on the LSUN image dataset based on pixel-wise reconstruction loss, and propose an optimization criterion that encourages unsupervised learning of informative latent features.