Free-Form Variational Inference for Gaussian Process State-Space Models
This addresses inference challenges in Bayesian GPSSMs for researchers and practitioners in time-series modeling, though it appears incremental as an extension of variational methods.
The paper tackles the computational and statistical challenges of inference in Gaussian process state-space models by proposing a free-form variational inference method using stochastic gradient Hamiltonian Monte Carlo with inducing variables. The approach learns transition dynamics and latent states more accurately than competing methods on six real-world datasets.
Gaussian process state-space models (GPSSMs) provide a principled and flexible approach to modeling the dynamics of a latent state, which is observed at discrete-time points via a likelihood model. However, inference in GPSSMs is computationally and statistically challenging due to the large number of latent variables in the model and the strong temporal dependencies between them. In this paper, we propose a new method for inference in Bayesian GPSSMs, which overcomes the drawbacks of previous approaches, namely over-simplified assumptions, and high computational requirements. Our method is based on free-form variational inference via stochastic gradient Hamiltonian Monte Carlo within the inducing-variable formalism. Furthermore, by exploiting our proposed variational distribution, we provide a collapsed extension of our method where the inducing variables are marginalized analytically. We also showcase results when combining our framework with particle MCMC methods. We show that, on six real-world datasets, our approach can learn transition dynamics and latent states more accurately than competing methods.