Graphical Models and Exponential Families
This work provides a foundational classification for graphical models, which is incremental in building on existing exponential family theory to aid in model selection for statisticians and machine learning researchers.
The paper tackles the problem of classifying graphical models by their representation as subfamilies of exponential families, showing that undirected models without hidden variables are linear exponential families, directed and chain graphs without hidden variables are curved exponential families, and models with hidden variables are stratified exponential families. It also demonstrates how to generate constraints for Bayesian networks with hidden variables and discusses implications for model selection.
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks with several families of local distributions, are curved exponential families (CEFs) and graphical models with hidden variables are stratified exponential families (SEFs). An SEF is a finite union of CEFs satisfying a frontier condition. In addition, we illustrate how one can automatically generate independence and non-independence constraints on the distributions over the observable variables implied by a Bayesian network with hidden variables. The relevance of these results for model selection is examined.