Sparse Gaussian Processes with Spherical Harmonic Features
This work addresses the scalability problem for practitioners using Gaussian processes in large-scale regression and classification tasks, though it is incremental as it builds on existing variational Fourier features.
The authors tackled the computational inefficiency of sparse Gaussian processes by introducing a new inter-domain variational method using spherical harmonic features, which enabled fitting a regression model on a dataset with 6 million entries two orders of magnitude faster than standard sparse GPs while maintaining state-of-the-art accuracy.
We introduce a new class of inter-domain variational Gaussian processes (GP) where data is mapped onto the unit hypersphere in order to use spherical harmonic representations. Our inference scheme is comparable to variational Fourier features, but it does not suffer from the curse of dimensionality, and leads to diagonal covariance matrices between inducing variables. This enables a speed-up in inference, because it bypasses the need to invert large covariance matrices. Our experiments show that our model is able to fit a regression model for a dataset with 6 million entries two orders of magnitude faster compared to standard sparse GPs, while retaining state of the art accuracy. We also demonstrate competitive performance on classification with non-conjugate likelihoods.