Kernel Implicit Variational Inference
This work addresses a bottleneck in variational inference for practitioners dealing with complex models, though it appears incremental as it builds on existing implicit distribution methods.
The paper tackles the challenge of applying implicit variational posteriors to models with high-dimensional latent variables, such as Bayesian neural networks, by introducing Kernel Implicit Variational Inference, which successfully demonstrates promising results on regression and classification tasks.
Recent progress in variational inference has paid much attention to the flexibility of variational posteriors. One promising direction is to use implicit distributions, i.e., distributions without tractable densities as the variational posterior. However, existing methods on implicit posteriors still face challenges of noisy estimation and computational infeasibility when applied to models with high-dimensional latent variables. In this paper, we present a new approach named Kernel Implicit Variational Inference that addresses these challenges. As far as we know, for the first time implicit variational inference is successfully applied to Bayesian neural networks, which shows promising results on both regression and classification tasks.