Parameter elimination in particle Gibbs sampling
This work addresses performance bottlenecks in Bayesian inference for state-space models, offering an incremental improvement for researchers in fields like ecology and epidemiology.
The paper tackles the challenge of improving particle Gibbs sampling performance in state-space models by marginalizing out conjugate parameters, which yields a non-Markovian model but scales linearly in time, and demonstrates its viability as an efficient inference backend in probabilistic programming with examples in ecology and epidemiology.
Bayesian inference in state-space models is challenging due to high-dimensional state trajectories. A viable approach is particle Markov chain Monte Carlo, combining MCMC and sequential Monte Carlo to form "exact approximations" to otherwise intractable MCMC methods. The performance of the approximation is limited to that of the exact method. We focus on particle Gibbs and particle Gibbs with ancestor sampling, improving their performance beyond that of the underlying Gibbs sampler (which they approximate) by marginalizing out one or more parameters. This is possible when the parameter prior is conjugate to the complete data likelihood. Marginalization yields a non-Markovian model for inference, but we show that, in contrast to the general case, this method still scales linearly in time. While marginalization can be cumbersome to implement, recent advances in probabilistic programming have enabled its automation. We demonstrate how the marginalized methods are viable as efficient inference backends in probabilistic programming, and demonstrate with examples in ecology and epidemiology.