Learning Integral Representations of Gaussian Processes
This work addresses computational bottlenecks in GP regression for machine learning practitioners, offering incremental improvements in efficiency.
The paper tackles the computational inefficiency of Gaussian processes (GPs) by proposing integral Gaussian processes (IGPs), which have sample paths within the reproducing kernel Hilbert space, leading to a regression algorithm that reduces computational complexity and prediction variance.
We propose a representation of Gaussian processes (GPs) based on powers of the integral operator defined by a kernel function, we call these stochastic processes integral Gaussian processes (IGPs). Sample paths from IGPs are functions contained within the reproducing kernel Hilbert space (RKHS) defined by the kernel function, in contrast sample paths from the standard GP are not functions within the RKHS. We develop computationally efficient non-parametric regression models based on IGPs. The main innovation in our regression algorithm is the construction of a low dimensional subspace that captures the information most relevant to explaining variation in the response. We use ideas from supervised dimension reduction to compute this subspace. The result of using the construction we propose involves significant improvements in the computational complexity of estimating kernel hyper-parameters as well as reducing the prediction variance.