Sparse Algorithms for Markovian Gaussian Processes
This work provides scalable inference methods for probabilistic models in real-world time series applications, representing an incremental advancement by extending existing algorithms with sparse techniques.
The paper tackles the challenge of scaling approximate Bayesian inference for large time series and spatio-temporal data by introducing sparse Markovian Gaussian processes, which combine inducing variables with Kalman filter-like recursions to achieve linear computational and memory scaling in inducing points, enabling parallel updates and stochastic optimization.
Approximate Bayesian inference methods that scale to very large datasets are crucial in leveraging probabilistic models for real-world time series. Sparse Markovian Gaussian processes combine the use of inducing variables with efficient Kalman filter-like recursions, resulting in algorithms whose computational and memory requirements scale linearly in the number of inducing points, whilst also enabling parallel parameter updates and stochastic optimisation. Under this paradigm, we derive a general site-based approach to approximate inference, whereby we approximate the non-Gaussian likelihood with local Gaussian terms, called sites. Our approach results in a suite of novel sparse extensions to algorithms from both the machine learning and signal processing literature, including variational inference, expectation propagation, and the classical nonlinear Kalman smoothers. The derived methods are suited to large time series, and we also demonstrate their applicability to spatio-temporal data, where the model has separate inducing points in both time and space.