Spectrum Gaussian Processes Based On Tunable Basis Functions
This work addresses computational bottlenecks in Gaussian processes for machine learning practitioners, offering an incremental improvement over existing spectral and variational methods.
The authors tackled the computational complexity of Gaussian processes by introducing tunable, local, and bounded basis functions inspired by quantum physics, achieving satisfactory or better results compared to state-of-the-art methods, particularly with poorly chosen kernel functions.
Spectral approximation and variational inducing learning for the Gaussian process are two popular methods to reduce computational complexity. However, in previous research, those methods always tend to adopt the orthonormal basis functions, such as eigenvectors in the Hilbert space, in the spectrum method, or decoupled orthogonal components in the variational framework. In this paper, inspired by quantum physics, we introduce a novel basis function, which is tunable, local and bounded, to approximate the kernel function in the Gaussian process. There are two adjustable parameters in these functions, which control their orthogonality to each other and limit their boundedness. And we conduct extensive experiments on open-source datasets to testify its performance. Compared to several state-of-the-art methods, it turns out that the proposed method can obtain satisfactory or even better results, especially with poorly chosen kernel functions.