Internal Incoherency Scores for Constraint-based Causal Discovery Algorithms
This work addresses the need for assumption validation in causal discovery methods, which is crucial for researchers and practitioners using algorithms like PC, but it is incremental as it builds on existing constraint-based frameworks.
The authors tackled the problem of validating assumptions in constraint-based causal discovery algorithms by proposing internal coherency scores to detect assumption violations and finite sample errors without requiring ground truth. They demonstrated these scores on the PC algorithm with simulated and real-world datasets, aiming to make such testing a standard tool.
Causal discovery aims to infer causal graphs from observational or experimental data. Methods such as the popular PC algorithm are based on conditional independence testing and utilize enabling assumptions, such as the faithfulness assumption, for their inferences. In practice, these assumptions, as well as the functional assumptions inherited from the chosen conditional independence test, are typically taken as a given and not further tested for their validity on the data. In this work, we propose internal coherency scores that allow testing for assumption violations and finite sample errors, whenever detectable without requiring ground truth or further statistical tests. We provide a complete classification of erroneous results, including a distinction between detectable and undetectable errors, and prove that the detectable erroneous results can be measured by our scores. We illustrate our coherency scores on the PC algorithm with simulated and real-world datasets, and envision that testing for internal coherency can become a standard tool in applying constraint-based methods, much like a suite of tests is used to validate the assumptions of classical regression analysis.