Multiresolution Kernel Approximation for Gaussian Process Regression
This addresses the problem of scaling Gaussian process regression to large datasets for researchers and practitioners in machine learning, representing a novel method for a known bottleneck.
The authors tackled the scalability issue of Gaussian process regression by introducing Multiresolution Kernel Approximation (MKA), a memory-efficient direct method that approximates kernel matrices effectively across broad bandwidths, enabling easier computation of inverses and determinants.
Gaussian process regression generally does not scale to beyond a few thousands data points without applying some sort of kernel approximation method. Most approximations focus on the high eigenvalue part of the spectrum of the kernel matrix, $K$, which leads to bad performance when the length scale of the kernel is small. In this paper we introduce Multiresolution Kernel Approximation (MKA), the first true broad bandwidth kernel approximation algorithm. Important points about MKA are that it is memory efficient, and it is a direct method, which means that it also makes it easy to approximate $K^{-1}$ and $\mathop{\textrm{det}}(K)$.