Distribution Matching in Variational Inference
This work addresses limitations in generative models for researchers and practitioners, but it is incremental as it analyzes existing hybrids without proposing a new solution.
The paper exposes that Variational Autoencoders (VAEs) fail to learn marginal distributions in latent and visible spaces due to conditional distribution matching, and finds that VAE-GAN hybrids are limited in scalability, evaluation, and inference without improving generation quality over GANs.
With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations. In this paper, we expose the limitations of Variational Autoencoders (VAEs), which consistently fail to learn marginal distributions in both latent and visible spaces. We show this to be a consequence of learning by matching conditional distributions, and the limitations of explicit model and posterior distributions. It is popular to consider Generative Adversarial Networks (GANs) as a means of overcoming these limitations, leading to hybrids of VAEs and GANs. We perform a large-scale evaluation of several VAE-GAN hybrids and analyze the implications of class probability estimation for learning distributions. While promising, we conclude that at present, VAE-GAN hybrids have limited applicability: they are harder to scale, evaluate, and use for inference compared to VAEs; and they do not improve over the generation quality of GANs.