Learning Convex Inference of Marginals
This addresses difficulties in probabilistic inference for machine learning practitioners, but appears incremental as it builds on free energy approximations.
The paper tackles the problem of approximate inference in graphical models by defining inference as convex minimization and learning directly via empirical risk on marginal prediction accuracy.
Graphical models trained using maximum likelihood are a common tool for probabilistic inference of marginal distributions. However, this approach suffers difficulties when either the inference process or the model is approximate. In this paper, the inference process is first defined to be the minimization of a convex function, inspired by free energy approximations. Learning is then done directly in terms of the performance of the inference process at univariate marginal prediction. The main novelty is that this is a direct minimization of emperical risk, where the risk measures the accuracy of predicted marginals.