Graphical model inference: Sequential Monte Carlo meets deterministic approximations
This work addresses the trade-off between bias and computational cost in probabilistic inference for researchers and practitioners in machine learning, offering an incremental improvement by combining existing techniques.
The paper tackled the problem of approximate inference in probabilistic graphical models by bridging deterministic and stochastic methods, resulting in a sequential Monte Carlo algorithm that leverages deterministic outputs to correct biases and shows clear improvements over baseline deterministic methods and plain SMC.
Approximate inference in probabilistic graphical models (PGMs) can be grouped into deterministic methods and Monte-Carlo-based methods. The former can often provide accurate and rapid inferences, but are typically associated with biases that are hard to quantify. The latter enjoy asymptotic consistency, but can suffer from high computational costs. In this paper we present a way of bridging the gap between deterministic and stochastic inference. Specifically, we suggest an efficient sequential Monte Carlo (SMC) algorithm for PGMs which can leverage the output from deterministic inference methods. While generally applicable, we show explicitly how this can be done with loopy belief propagation, expectation propagation, and Laplace approximations. The resulting algorithm can be viewed as a post-correction of the biases associated with these methods and, indeed, numerical results show clear improvements over the baseline deterministic methods as well as over "plain" SMC.