Asymptotic Model Selection for Directed Networks with Hidden Variables
This work provides a theoretical tool for statisticians and machine learning researchers to select models in directed networks with hidden variables, but it is incremental as it builds on existing BIC methodology.
The authors extended the Bayesian Information Criterion (BIC) to Bayesian networks with hidden variables for model selection in large samples, arguing that model complexity should be based on the rank of a Jacobian matrix and computing dimensions for specific networks like naive Bayes.
We extend the Bayesian Information Criterion (BIC), an asymptotic approximation for the marginal likelihood, to Bayesian networks with hidden variables. This approximation can be used to select models given large samples of data. The standard BIC as well as our extension punishes the complexity of a model according to the dimension of its parameters. We argue that the dimension of a Bayesian network with hidden variables is the rank of the Jacobian matrix of the transformation between the parameters of the network and the parameters of the observable variables. We compute the dimensions of several networks including the naive Bayes model with a hidden root node.