Constant-Time Predictive Distributions for Gaussian Processes
This work addresses a key computational limitation for researchers and practitioners using Gaussian processes, enabling faster uncertainty quantification in applications like machine learning and statistics.
The paper tackles the computational bottleneck of posterior covariance estimation and sampling in Gaussian process regression by introducing LOVE, a method using the Lanczos algorithm to approximate the predictive covariance matrix, achieving up to 2,000 times faster covariance computations and 18,000 times faster sampling without accuracy loss.
One of the most compelling features of Gaussian process (GP) regression is its ability to provide well-calibrated posterior distributions. Recent advances in inducing point methods have sped up GP marginal likelihood and posterior mean computations, leaving posterior covariance estimation and sampling as the remaining computational bottlenecks. In this paper we address these shortcomings by using the Lanczos algorithm to rapidly approximate the predictive covariance matrix. Our approach, which we refer to as LOVE (LanczOs Variance Estimates), substantially improves time and space complexity. In our experiments, LOVE computes covariances up to 2,000 times faster and draws samples 18,000 times faster than existing methods, all without sacrificing accuracy.