Choosing DAG Models Using Markov and Minimal Edge Count in the Absence of Ground Truth
This work addresses the problem of choosing approximately correct causal models for researchers and practitioners in causal inference, particularly when ground truth is unavailable.
This paper introduces the Markov Checker, a nonparametric statistical test for the Markov condition in DAG/CPDAG models, and the Cross-Algorithm Frugality Search (CAFS), a method for rejecting DAG models that fail the Markov Checker or are not edge minimal. The CAFS procedure is shown in simulations to pick approximately correct models without requiring ground truth.
We give a novel nonparametric pointwise consistent statistical test (the Markov Checker) of the Markov condition for directed acyclic graph (DAG) or completed partially directed acyclic graph (CPDAG) models given a dataset. We also introduce the Cross-Algorithm Frugality Search (CAFS) for rejecting DAG models that either do not pass the Markov Checker test or that are not edge minimal. Edge minimality has been used previously by Raskutti and Uhler as a nonparametric simplicity criterion, though CAFS readily generalizes to other simplicity conditions. Reference to the ground truth is not necessary for CAFS, so it is useful for finding causal structure learning algorithms and tuning parameter settings that output causal models that are approximately true from a given data set. We provide a software tool for this analysis that is suitable for even quite large or dense models, provided a suitably fast pointwise consistent test of conditional independence is available. In addition, we show in simulation that the CAFS procedure can pick approximately correct models without knowing the ground truth.