Actually Sparse Variational Gaussian Processes
This work addresses computational bottlenecks for researchers and practitioners using GPs in large-scale spatial data applications, representing an incremental improvement over existing sparse GP methods.
The authors tackled the computational and memory inefficiency of Gaussian processes (GPs) for large datasets with many inducing variables, such as low-lengthscale spatial data, by proposing a new class of inter-domain variational GP using compactly supported B-spline basis functions. The result was a method that enables modeling fast-varying spatial phenomena with tens of thousands of inducing variables, significantly speeding up matrix operations and reducing memory footprint where previous approaches failed.
Gaussian processes (GPs) are typically criticised for their unfavourable scaling in both computational and memory requirements. For large datasets, sparse GPs reduce these demands by conditioning on a small set of inducing variables designed to summarise the data. In practice however, for large datasets requiring many inducing variables, such as low-lengthscale spatial data, even sparse GPs can become computationally expensive, limited by the number of inducing variables one can use. In this work, we propose a new class of inter-domain variational GP, constructed by projecting a GP onto a set of compactly supported B-spline basis functions. The key benefit of our approach is that the compact support of the B-spline basis functions admits the use of sparse linear algebra to significantly speed up matrix operations and drastically reduce the memory footprint. This allows us to very efficiently model fast-varying spatial phenomena with tens of thousands of inducing variables, where previous approaches failed.