Integrated Variational Fourier Features for Fast Spatial Modelling with Gaussian Processes
This work addresses computational bottlenecks in Gaussian processes for spatial modeling, offering an incremental improvement by broadening kernel compatibility while maintaining efficiency.
The authors tackled the problem of scaling Gaussian process inference for spatial modeling by proposing integrated Fourier features, which extend fast $O(M^3)$ cost methods to a broad class of stationary covariance functions, achieving practical speedup in synthetic and real-world tasks.
Sparse variational approximations are popular methods for scaling up inference and learning in Gaussian processes to larger datasets. For $N$ training points, exact inference has $O(N^3)$ cost; with $M \ll N$ features, state of the art sparse variational methods have $O(NM^2)$ cost. Recently, methods have been proposed using more sophisticated features; these promise $O(M^3)$ cost, with good performance in low dimensional tasks such as spatial modelling, but they only work with a very limited class of kernels, excluding some of the most commonly used. In this work, we propose integrated Fourier features, which extends these performance benefits to a very broad class of stationary covariance functions. We motivate the method and choice of parameters from a convergence analysis and empirical exploration, and show practical speedup in synthetic and real world spatial regression tasks.