Variational Approximations between Mean Field Theory and the Junction Tree Algorithm
This work addresses the challenge of efficient inference in probabilistic graphical models for researchers and practitioners, offering an incremental improvement over existing variational methods.
The paper tackles the problem of improving variational approximations in probabilistic graphical models by extending the mean field method to use cluster potentials, bridging the gap between mean field theory and the exact junction tree algorithm. It results in generalized mean field equations and rules for simplifying the approximating distribution without loss of quality.
Recently, variational approximations such as the mean field approximation have received much interest. We extend the standard mean field method by using an approximating distribution that factorises into cluster potentials. This includes undirected graphs, directed acyclic graphs and junction trees. We derive generalized mean field equations to optimize the cluster potentials. We show that the method bridges the gap between the standard mean field approximation and the exact junction tree algorithm. In addition, we address the problem of how to choose the graphical structure of the approximating distribution. From the generalised mean field equations we derive rules to simplify the structure of the approximating distribution in advance without affecting the quality of the approximation. We also show how the method fits into some other variational approximations that are currently popular.