The Neural Moving Average Model for Scalable Variational Inference of State Space Models
This addresses the problem of computational efficiency in Bayesian inference for time series data, though it is incremental as it extends existing variational inference methods to a new model type.
The paper tackled the challenge of scaling variational inference to state space models for time series data, proposing a neural moving average model that enables mini-batch training and achieves accurate parameter estimation quickly, as demonstrated on various models like autoregressive and stochastic volatility.
Variational inference has had great success in scaling approximate Bayesian inference to big data by exploiting mini-batch training. To date, however, this strategy has been most applicable to models of independent data. We propose an extension to state space models of time series data based on a novel generative model for latent temporal states: the neural moving average model. This permits a subsequence to be sampled without drawing from the entire distribution, enabling training iterations to use mini-batches of the time series at low computational cost. We illustrate our method on autoregressive, Lotka-Volterra, FitzHugh-Nagumo and stochastic volatility models, achieving accurate parameter estimation in a short time.