Wasserstein-Splitting Gaussian Process Regression for Heterogeneous Online Bayesian Inference
This addresses scalability and adaptability problems for Bayesian inference in machine learning, though it appears incremental as it builds on existing GP and variational methods.
The paper tackled scalability and performance issues of Gaussian processes for large, non-stationary, or heterogeneous data by introducing a method that uses variational free energy approximations with online expectation propagation and a Wasserstein-based splitting step to create an ensemble of sparse GPs, resulting in improved adaptability and incremental updates.
Gaussian processes (GPs) are a well-known nonparametric Bayesian inference technique, but they suffer from scalability problems for large sample sizes, and their performance can degrade for non-stationary or spatially heterogeneous data. In this work, we seek to overcome these issues through (i) employing variational free energy approximations of GPs operating in tandem with online expectation propagation steps; and (ii) introducing a local splitting step which instantiates a new GP whenever the posterior distribution changes significantly as quantified by the Wasserstein metric over posterior distributions. Over time, then, this yields an ensemble of sparse GPs which may be updated incrementally, and adapts to locality, heterogeneity, and non-stationarity in training data.