Scalable Training of Inference Networks for Gaussian-Process Models
This addresses the problem of scalable inference for researchers and practitioners using Gaussian processes, though it appears incremental as it builds on existing network-based approximations.
The paper tackles the computational challenge of inference in Gaussian process models for large datasets by proposing a scalable training algorithm for stochastic inference networks, achieving performance comparable or superior to existing sparse variational GP methods.
Inference in Gaussian process (GP) models is computationally challenging for large data, and often difficult to approximate with a small number of inducing points. We explore an alternative approximation that employs stochastic inference networks for a flexible inference. Unfortunately, for such networks, minibatch training is difficult to be able to learn meaningful correlations over function outputs for a large dataset. We propose an algorithm that enables such training by tracking a stochastic, functional mirror-descent algorithm. At each iteration, this only requires considering a finite number of input locations, resulting in a scalable and easy-to-implement algorithm. Empirical results show comparable and, sometimes, superior performance to existing sparse variational GP methods.