Unbiased Implicit Variational Inference
This work addresses the problem of improving variational inference for practitioners in machine learning by offering a more expressive and efficient method, though it is incremental as it builds on existing variational inference frameworks.
The paper tackles the challenge of limited expressiveness in variational inference by introducing Unbiased Implicit Variational Inference (UIVI), which uses deep neural networks to define an implicit variational distribution and directly optimizes the evidence lower bound (ELBO), resulting in tighter ELBO and better predictive performance than existing methods at similar computational costs.
We develop unbiased implicit variational inference (UIVI), a method that expands the applicability of variational inference by defining an expressive variational family. UIVI considers an implicit variational distribution obtained in a hierarchical manner using a simple reparameterizable distribution whose variational parameters are defined by arbitrarily flexible deep neural networks. Unlike previous works, UIVI directly optimizes the evidence lower bound (ELBO) rather than an approximation to the ELBO. We demonstrate UIVI on several models, including Bayesian multinomial logistic regression and variational autoencoders, and show that UIVI achieves both tighter ELBO and better predictive performance than existing approaches at a similar computational cost.