Gaussian variational approximation for high-dimensional state space models
This work provides a scalable variational inference method for high-dimensional state space models, which is incremental as it builds on existing Gaussian approximations but introduces a specific parameterization to handle large-scale problems in fields like ecology and finance.
The authors tackled the challenge of approximating posterior densities in high-dimensional state space models by proposing a Gaussian variational approximation with a dynamic factor model parameterization, which reduces the number of covariance parameters and maintains temporal structure. They demonstrated its effectiveness for prediction in two high-dimensional applications, including a spatio-temporal model for species spread and a financial volatility model, where it addresses difficulties faced by Markov chain Monte Carlo sampling.
Our article considers a Gaussian variational approximation of the posterior density in a high-dimensional state space model. The variational parameters to be optimized are the mean vector and the covariance matrix of the approximation. The number of parameters in the covariance matrix grows as the square of the number of model parameters, so it is necessary to find simple yet effective parameterizations of the covariance structure when the number of model parameters is large. We approximate the joint posterior distribution over the high-dimensional state vectors by a dynamic factor model, having Markovian time dependence and a factor covariance structure for the states. This gives a reduced description of the dependence structure for the states, as well as a temporal conditional independence structure similar to that in the true posterior. The usefulness of the approach is illustrated for prediction in two high-dimensional applications that are challenging for Markov chain Monte Carlo sampling. The first is a spatio-temporal model for the spread of the Eurasian Collared-Dove across North America; the second is a Wishart-based multivariate stochastic volatility model for financial returns.