Conditions Under Which Conditional Independence and Scoring Methods Lead to Identical Selection of Bayesian Network Models
This clarifies a foundational issue in Bayesian network inference, potentially simplifying methodology for researchers in machine learning and statistics.
The paper tackles the perceived distinction between conditional independence tests and scoring methods for selecting Bayesian network structures, showing that for complete data and a given node ordering, cross entropy methods for conditional independence are mathematically identical to goodness-of-fit logarithmic scores.
It is often stated in papers tackling the task of inferring Bayesian network structures from data that there are these two distinct approaches: (i) Apply conditional independence tests when testing for the presence or otherwise of edges; (ii) Search the model space using a scoring metric. Here I argue that for complete data and a given node ordering this division is a myth, by showing that cross entropy methods for checking conditional independence are mathematically identical to methods based upon discriminating between models by their overall goodness-of-fit logarithmic scores.