Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models
This addresses the computational bottleneck in Gaussian process inference for practitioners, offering significant speed improvements.
The paper tackles inference in non-conjugate Gaussian process models by proposing automated augmented conjugate inference, which constructs auxiliary variables to enable conditional conjugacy, resulting in methods that are up to two orders of magnitude faster and more robust than existing state-of-the-art approaches.
We propose automated augmented conjugate inference, a new inference method for non-conjugate Gaussian processes (GP) models. Our method automatically constructs an auxiliary variable augmentation that renders the GP model conditionally conjugate. Building on the conjugate structure of the augmented model, we develop two inference methods. First, a fast and scalable stochastic variational inference method that uses efficient block coordinate ascent updates, which are computed in closed form. Second, an asymptotically correct Gibbs sampler that is useful for small datasets. Our experiments show that our method are up two orders of magnitude faster and more robust than existing state-of-the-art black-box methods.