Variational Implicit Processes
This provides a scalable inference method for complex implicit priors, benefiting researchers and practitioners in Bayesian machine learning, though it is incremental as it builds on existing variational and implicit approaches.
The paper tackles the challenge of flexible implicit priors over functions by introducing implicit processes (IPs) and develops variational implicit processes (VIPs), an efficient approximate inference algorithm using generalized wake-sleep updates, which in experiments yields better uncertainty estimates and lower errors compared to existing methods for models like Bayesian neural networks and Gaussian processes.
We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables. IPs are therefore highly flexible implicit priors over functions, with examples including data simulators, Bayesian neural networks and non-linear transformations of stochastic processes. A novel and efficient approximate inference algorithm for IPs, namely the variational implicit processes (VIPs), is derived using generalised wake-sleep updates. This method returns simple update equations and allows scalable hyper-parameter learning with stochastic optimization. Experiments show that VIPs return better uncertainty estimates and lower errors over existing inference methods for challenging models such as Bayesian neural networks, and Gaussian processes.