MCMC for Variationally Sparse Gaussian Processes
This addresses efficiency and flexibility issues in Gaussian process models for probabilistic machine learning practitioners, representing an incremental improvement by combining existing techniques.
The paper tackled the challenges of scaling Gaussian processes to large datasets, handling non-Gaussian likelihoods, and estimating covariance parameters by developing a variational approximation that is sparse in function support. The result is a Hybrid Monte-Carlo sampling scheme enabling efficient non-Gaussian approximations over function values and parameters, with computational efficiency based on inducing-point sparse GPs.
Gaussian process (GP) models form a core part of probabilistic machine learning. Considerable research effort has been made into attacking three issues with GP models: how to compute efficiently when the number of data is large; how to approximate the posterior when the likelihood is not Gaussian and how to estimate covariance function parameter posteriors. This paper simultaneously addresses these, using a variational approximation to the posterior which is sparse in support of the function but otherwise free-form. The result is a Hybrid Monte-Carlo sampling scheme which allows for a non-Gaussian approximation over the function values and covariance parameters simultaneously, with efficient computations based on inducing-point sparse GPs. Code to replicate each experiment in this paper will be available shortly.