Bayesian Random Fields: The Bethe-Laplace Approximation
This work addresses the problem of Bayesian inference in undirected models for researchers in computer vision and text modeling, representing an incremental advancement in a nascent field.
The authors tackled the challenge of approximating posterior distributions over parameters in undirected graphical models, such as conditional random fields, by proposing a new method based on the Laplace approximation with linear response computation via loopy belief propagation, and validated it with experiments on real-world data.
While learning the maximum likelihood value of parameters of an undirected graphical model is hard, modelling the posterior distribution over parameters given data is harder. Yet, undirected models are ubiquitous in computer vision and text modelling (e.g. conditional random fields). But where Bayesian approaches for directed models have been very successful, a proper Bayesian treatment of undirected models in still in its infant stages. We propose a new method for approximating the posterior of the parameters given data based on the Laplace approximation. This approximation requires the computation of the covariance matrix over features which we compute using the linear response approximation based in turn on loopy belief propagation. We develop the theory for conditional and 'unconditional' random fields with or without hidden variables. In the conditional setting we introduce a new variant of bagging suitable for structured domains. Here we run the loopy max-product algorithm on a 'super-graph' composed of graphs for individual models sampled from the posterior and connected by constraints. Experiments on real world data validate the proposed methods.