High-Dimensional Gaussian Process Regression with Soft Kernel Interpolation
This work addresses scalability issues in Gaussian Process regression for high-dimensional data, but it is incremental as it builds on existing methods like SKI and variational approaches.
The authors tackled the problem of scaling Gaussian Process regression to high-dimensional datasets by introducing Soft Kernel Interpolation (SoftKI), which combines aspects of Structured Kernel Interpolation and variational methods to overcome dimensionality challenges, showing competitive performance with other approximated GP methods for modest dimensionalities around 10.
We introduce Soft Kernel Interpolation (SoftKI), a method that combines aspects of Structured Kernel Interpolation (SKI) and variational inducing point methods, to achieve scalable Gaussian Process (GP) regression on high-dimensional datasets. SoftKI approximates a kernel via softmax interpolation from a smaller number of interpolation points learned by optimizing a combination of the SoftKI marginal log-likelihood (MLL), and when needed, an approximate MLL for improved numerical stability. Consequently, it can overcome the dimensionality scaling challenges that SKI faces when interpolating from a dense and static lattice while retaining the flexibility of variational methods to adapt inducing points to the dataset. We demonstrate the effectiveness of SoftKI across various examples and show that it is competitive with other approximated GP methods when the data dimensionality is modest (around 10).