Worst-case Optimal Submodular Extensions for Marginal Estimation
This work addresses a bottleneck in probabilistic graphical models for researchers and practitioners by improving marginal estimation accuracy, though it is incremental as it builds on existing submodular extension and LP relaxation frameworks.
The paper tackles the problem of computing approximate marginals via variational inference by identifying worst-case optimal submodular extensions for energy functions, establishing optimality for Potts and metric labeling models and enabling efficient marginal computation for dense CRF models. It shows comparable upper bounds on the log-partition function to tree-reweighted message passing in feasible cases and provides the first practical algorithm for dense CRF models.
Submodular extensions of an energy function can be used to efficiently compute approximate marginals via variational inference. The accuracy of the marginals depends crucially on the quality of the submodular extension. To identify the best possible extension, we show an equivalence between the submodular extensions of the energy and the objective functions of linear programming (LP) relaxations for the corresponding MAP estimation problem. This allows us to (i) establish the worst-case optimality of the submodular extension for Potts model used in the literature; (ii) identify the worst-case optimal submodular extension for the more general class of metric labeling; and (iii) efficiently compute the marginals for the widely used dense CRF model with the help of a recently proposed Gaussian filtering method. Using synthetic and real data, we show that our approach provides comparable upper bounds on the log-partition function to those obtained using tree-reweighted message passing (TRW) in cases where the latter is computationally feasible. Importantly, unlike TRW, our approach provides the first practical algorithm to compute an upper bound on the dense CRF model.