State Space Gaussian Processes with Non-Gaussian Likelihood
This work addresses a computational bottleneck for researchers and practitioners in machine learning and statistics, enabling faster inference in GP models with non-Gaussian likelihoods, though it is incremental as it builds on existing state space and inference methods.
The paper tackles the computational inefficiency of Gaussian process (GP) modeling with non-Gaussian likelihoods by leveraging state space methods, achieving linear time and memory complexity of O(n) for one-dimensional models.
We provide a comprehensive overview and tooling for GP modeling with non-Gaussian likelihoods using state space methods. The state space formulation allows for solving one-dimensional GP models in $\mathcal{O}(n)$ time and memory complexity. While existing literature has focused on the connection between GP regression and state space methods, the computational primitives allowing for inference using general likelihoods in combination with the Laplace approximation (LA), variational Bayes (VB), and assumed density filtering (ADF, a.k.a. single-sweep expectation propagation, EP) schemes has been largely overlooked. We present means of combining the efficient $\mathcal{O}(n)$ state space methodology with existing inference methods. We extend existing methods, and provide unifying code implementing all approaches.