EigenGP: Gaussian Process Models with Adaptive Eigenfunctions
This work addresses computational bottlenecks in Gaussian processes for machine learning practitioners, though it is incremental as it builds on existing sparse Bayesian finite models.
The paper tackles the high computational cost of Gaussian process inference for big data by proposing EigenGP, a Bayesian approach that learns adaptive eigenfunctions and prior precisions, achieving improved predictive performance over alternative sparse GP methods and relevance vector machine.
Gaussian processes (GPs) provide a nonparametric representation of functions. However, classical GP inference suffers from high computational cost for big data. In this paper, we propose a new Bayesian approach, EigenGP, that learns both basis dictionary elements--eigenfunctions of a GP prior--and prior precisions in a sparse finite model. It is well known that, among all orthogonal basis functions, eigenfunctions can provide the most compact representation. Unlike other sparse Bayesian finite models where the basis function has a fixed form, our eigenfunctions live in a reproducing kernel Hilbert space as a finite linear combination of kernel functions. We learn the dictionary elements--eigenfunctions--and the prior precisions over these elements as well as all the other hyperparameters from data by maximizing the model marginal likelihood. We explore computational linear algebra to simplify the gradient computation significantly. Our experimental results demonstrate improved predictive performance of EigenGP over alternative sparse GP methods as well as relevance vector machine.