The DLR Hierarchy of Approximate Inference
This work provides a theoretical framework for approximate inference in probabilistic models, which is incremental as it builds on existing DLR equations and algorithms.
The authors tackled the problem of approximate inference by proposing a DLR-based hierarchy that unifies existing algorithms and introduces new ones, showing that extrema of the Bethe free energy correspond to approximate DLR solutions and demonstrating a connection to Gibbs sampling, with experiments on spin-glass problems indicating higher-level algorithms yield more accurate but less stable results.
We propose a hierarchy for approximate inference based on the Dobrushin, Lanford, Ruelle (DLR) equations. This hierarchy includes existing algorithms, such as belief propagation, and also motivates novel algorithms such as factorized neighbors (FN) algorithms and variants of mean field (MF) algorithms. In particular, we show that extrema of the Bethe free energy correspond to approximate solutions of the DLR equations. In addition, we demonstrate a close connection between these approximate algorithms and Gibbs sampling. Finally, we compare and contrast various of the algorithms in the DLR hierarchy on spin-glass problems. The experiments show that algorithms higher up in the hierarchy give more accurate results when they converge but tend to be less stable.