Distributed Gaussian Processes
This addresses the computational bottleneck in Gaussian processes for large-scale regression, offering a practical solution for distributed computing environments.
The paper tackles scaling Gaussian processes to large datasets by introducing the robust Bayesian Committee Machine (rBCM), a distributed product-of-experts model that enables efficient parallelization and handles arbitrarily large datasets with sufficient computing resources.
To scale Gaussian processes (GPs) to large data sets we introduce the robust Bayesian Committee Machine (rBCM), a practical and scalable product-of-experts model for large-scale distributed GP regression. Unlike state-of-the-art sparse GP approximations, the rBCM is conceptually simple and does not rely on inducing or variational parameters. The key idea is to recursively distribute computations to independent computational units and, subsequently, recombine them to form an overall result. Efficient closed-form inference allows for straightforward parallelisation and distributed computations with a small memory footprint. The rBCM is independent of the computational graph and can be used on heterogeneous computing infrastructures, ranging from laptops to clusters. With sufficient computing resources our distributed GP model can handle arbitrarily large data sets.