$α$ Belief Propagation as Fully Factorized Approximation
This work addresses a fundamental limitation in probabilistic inference for machine learning and AI, offering an interpretable improvement over standard methods.
The paper tackles the problem of poor performance and limited interpretability of belief propagation in graphs with loops by introducing α belief propagation, which minimizes a localized α-divergence. In MAP inference tests, α-BP significantly outperforms loopy BP, even in fully-connected graphs.
Belief propagation (BP) can do exact inference in loop-free graphs, but its performance could be poor in graphs with loops, and the understanding of its solution is limited. This work gives an interpretable belief propagation rule that is actually minimization of a localized $α$-divergence. We term this algorithm as $α$ belief propagation ($α$-BP). The performance of $α$-BP is tested in MAP (maximum a posterior) inference problems, where $α$-BP can outperform (loopy) BP by a significant margin even in fully-connected graphs.