Product Kernel Interpolation for Scalable Gaussian Processes
This work addresses scalability issues in Gaussian processes for machine learning practitioners, offering a significant improvement over existing methods.
The paper tackles the curse of dimensionality in Gaussian process inference by developing a product kernel interpolation technique, resulting in linear runtime with dimension instead of exponential and achieving state-of-the-art asymptotic complexity for multi-task GPs.
Recent work shows that inference for Gaussian processes can be performed efficiently using iterative methods that rely only on matrix-vector multiplications (MVMs). Structured Kernel Interpolation (SKI) exploits these techniques by deriving approximate kernels with very fast MVMs. Unfortunately, such strategies suffer badly from the curse of dimensionality. We develop a new technique for MVM based learning that exploits product kernel structure. We demonstrate that this technique is broadly applicable, resulting in linear rather than exponential runtime with dimension for SKI, as well as state-of-the-art asymptotic complexity for multi-task GPs.