Self-Guided Belief Propagation -- A Homotopy Continuation Method
This work addresses the challenge of non-convergence and accuracy limitations in belief propagation for graphical models, offering a method that is incremental but provides practical improvements for researchers and practitioners in machine learning and probabilistic inference.
The paper tackles the problem of improving belief propagation (BP) for probabilistic inference by proposing self-guided belief propagation (SBP), a homotopy continuation method that gradually incorporates pairwise potentials, which converges to a unique solution and increases accuracy without extra computational cost, empirically showing superiority in accuracy when BP converges and providing stable solutions when BP fails.
Belief propagation (BP) is a popular method for performing probabilistic inference on graphical models. In this work, we enhance BP and propose self-guided belief propagation (SBP) that incorporates the pairwise potentials only gradually. This homotopy continuation method converges to a unique solution and increases the accuracy without increasing the computational burden. We provide a formal analysis to demonstrate that SBP finds the global optimum of the Bethe approximation for attractive models where all variables favor the same state. Moreover, we apply SBP to various graphs with random potentials and empirically show that: (i) SBP is superior in terms of accuracy whenever BP converges, and (ii) SBP obtains a unique, stable, and accurate solution whenever BP does not converge.