Amortized Variational Inference for Deep Gaussian Processes
This work addresses the problem of efficient and expressive inference for deep Gaussian processes, which is incremental as it builds on existing variational techniques to enhance model flexibility.
The paper tackles the challenge of tractable inference in deep Gaussian processes (DGPs) by introducing amortized variational inference, which learns an inference function to map observations to variational parameters, resulting in improved expressivity with fewer inducing variables. Experimental results show it performs similarly or better than previous methods at reduced computational cost.
Gaussian processes (GPs) are Bayesian nonparametric models for function approximation with principled predictive uncertainty estimates. Deep Gaussian processes (DGPs) are multilayer generalizations of GPs that can represent complex marginal densities as well as complex mappings. As exact inference is either computationally prohibitive or analytically intractable in GPs and extensions thereof, some existing methods resort to variational inference (VI) techniques for tractable approximations. However, the expressivity of conventional approximate GP models critically relies on independent inducing variables that might not be informative enough for some problems. In this work we introduce amortized variational inference for DGPs, which learns an inference function that maps each observation to variational parameters. The resulting method enjoys a more expressive prior conditioned on fewer input dependent inducing variables and a flexible amortized marginal posterior that is able to model more complicated functions. We show with theoretical reasoning and experimental results that our method performs similarly or better than previous approaches at less computational cost.