Adaptive Variational Inference in Probabilistic Graphical Models: Beyond Bethe, Tree-Reweighted, and Convex Free Energies
This work addresses limitations in variational inference for probabilistic graphical models, offering an incremental improvement over existing methods like Bethe and tree-reweighted approximations.
The paper tackles the problem of variational inference in probabilistic graphical models failing on complex, highly interactive models by proposing adaptive approximations that automatically adjust to a given model, demonstrating effectiveness on difficult problems.
Variational inference in probabilistic graphical models aims to approximate fundamental quantities such as marginal distributions and the partition function. Popular approaches are the Bethe approximation, tree-reweighted, and other types of convex free energies. These approximations are efficient but can fail if the model is complex and highly interactive. In this work, we analyze two classes of approximations that include the above methods as special cases: first, if the model parameters are changed; and second, if the entropy approximation is changed. We discuss benefits and drawbacks of either approach, and deduce from this analysis how a free energy approximation should ideally be constructed. Based on our observations, we propose approximations that automatically adapt to a given model and demonstrate their effectiveness for a range of difficult problems.