Optimization of Structured Mean Field Objectives
This work addresses computational bottlenecks in approximate inference for machine learning, offering incremental improvements for researchers in probabilistic graphical models.
The paper tackles the problem of optimizing structured mean field objectives in intractable graphical models by analyzing the computational hardness based on a graph property called v-acyclicity, and it introduces new algorithms that enable faster optimization or inference in harder cases, with empirical comparisons provided.
In intractable, undirected graphical models, an intuitive way of creating structured mean field approximations is to select an acyclic tractable subgraph. We show that the hardness of computing the objective function and gradient of the mean field objective qualitatively depends on a simple graph property. If the tractable subgraph has this property- we call such subgraphs v-acyclic-a very fast block coordinate ascent algorithm is possible. If not, optimization is harder, but we show a new algorithm based on the construction of an auxiliary exponential family that can be used to make inference possible in this case as well. We discuss the advantages and disadvantages of each regime and compare the algorithms empirically.