Dynamic Trees: A Structured Variational Method Giving Efficient Propagation Rules
This work addresses inference challenges in probabilistic graphical models for researchers in machine learning, offering an incremental improvement over existing approximation methods.
The paper tackles the intractable exact inference in dynamic trees by proposing a structured variational method that approximates the posterior with another dynamic tree, resulting in efficient propagation rules that outperform mean field and loopy propagation in both efficiency and accuracy on a toy problem.
Dynamic trees are mixtures of tree structured belief networks. They solve some of the problems of fixed tree networks at the cost of making exact inference intractable. For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorized distribution over node states. Such a distribution seems unlickely in the posterior, as nodes are highly correlated in the prior. Here a structured variational approach is used, where the posterior distribution over the non-evidential nodes is itself approximated by a dynamic tree. It turns out that this form can be used tractably and efficiently. The result is a set of update rules which can propagate information through the network to obtain both a full variational approximation, and the relevant marginals. The progagtion rules are more efficient than the mean field approach and give noticeable quantitative and qualitative improvement in the inference. The marginals calculated give better approximations to the posterior than loopy propagation on a small toy problem.