Bayesian learning of forest and tree graphical models
This work addresses Bayesian inference for graphical models, offering incremental improvements for researchers in statistics and machine learning by optimizing algorithms for specific graph structures.
The paper tackles Bayesian structure learning for Gaussian graphical models by focusing on forests and trees, presenting corrected algorithms for non-decomposable graphs and adapted MCMC and stochastic shotgun search methods. Experiments show that SSS with trees performs well on tree or sparse graphs, outperforming decomposable graph methods in some cases, while MCMC on forests mixes poorly and on trees is slower than SSS.
In Bayesian learning of Gaussian graphical model structure, it is common to restrict attention to certain classes of graphs and approximate the posterior distribution by repeatedly moving from one graph to another, using MCMC or methods such as stochastic shotgun search (SSS). I give two corrected versions of an algorithm for non-decomposable graphs and discuss random graph distributions, in particular as prior distributions. The main topic of the thesis is Bayesian structure-learning with forests or trees. Restricting attention to these graphs can be justified using theorems on random graphs. I describe how to use the Chow$\unicode{x2013}$Liu algorithm and the Matrix Tree Theorem to find the MAP forest and certain quantities in the posterior distribution on trees. I give adapted versions of MCMC and SSS for approximating the posterior distribution for forests and trees, and systems for storing these graphs so that it is easy to choose moves to neighbouring graphs. Experiments show that SSS with trees does well when the true graph is a tree or sparse graph. SSS with trees or forests does better than SSS with decomposable graphs in certain cases. Graph priors improve detection of hubs but need large ranges of probabilities. MCMC on forests fails to mix well and MCMC on trees is slower than SSS. (For a longer abstract see the thesis.)