Accelerated training of Gaussian processes using banded square exponential covariances
This is an incremental improvement for researchers and practitioners using Gaussian processes in machine learning.
The authors tackled the computational inefficiency of training Gaussian processes by proposing a banded-matrix approximation to square-exponential covariance matrices, which reduces computational cost for likelihood approximation and shows efficiency compared to variational sparse GPs.
We propose a novel approach to computationally efficient GP training based on the observation that square-exponential (SE) covariance matrices contain several off-diagonal entries extremely close to zero. We construct a principled procedure to eliminate those entries to produce a \emph{banded}-matrix approximation to the original covariance, whose inverse and determinant can be computed at a reduced computational cost, thus contributing to an efficient approximation to the likelihood function. We provide a theoretical analysis of the proposed method to preserve the structure of the original covariance in the 1D setting with SE kernel, and validate its computational efficiency against the variational free energy approach to sparse GPs.