A Generalized Loop Correction Method for Approximate Inference in Graphical Models
This addresses a fundamental challenge in probabilistic inference for machine learning, offering an incremental improvement over prior methods.
The paper tackles the problem of inaccurate or non-convergent inference in loopy graphical models by introducing Generalized Loop Correction (GLC), which combines cavity distribution and region-based methods to improve accuracy, showing it is significantly more effective than existing correction schemes.
Belief Propagation (BP) is one of the most popular methods for inference in probabilistic graphical models. BP is guaranteed to return the correct answer for tree structures, but can be incorrect or non-convergent for loopy graphical models. Recently, several new approximate inference algorithms based on cavity distribution have been proposed. These methods can account for the effect of loops by incorporating the dependency between BP messages. Alternatively, region-based approximations (that lead to methods such as Generalized Belief Propagation) improve upon BP by considering interactions within small clusters of variables, thus taking small loops within these clusters into account. This paper introduces an approach, Generalized Loop Correction (GLC), that benefits from both of these types of loop correction. We show how GLC relates to these two families of inference methods, then provide empirical evidence that GLC works effectively in general, and can be significantly more accurate than both correction schemes.