Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery
This work identifies a fundamental performance constraint for researchers and practitioners in causal discovery, indicating that incremental improvements are insufficient and a paradigm shift is needed.
The paper tackles the problem of neural causal discovery methods' limited accuracy in uncovering causal structures, revealing that they cannot reliably distinguish between existing and non-existing causal relationships in finite samples, with experiments showing poor graph recovery even for small graphs and large sample sizes.
Neural causal discovery methods have recently improved in terms of scalability and computational efficiency. However, our systematic evaluation highlights significant room for improvement in their accuracy when uncovering causal structures. We identify a fundamental limitation: neural networks cannot reliably distinguish between existing and non-existing causal relationships in the finite sample regime. Our experiments reveal that neural networks, as used in contemporary causal discovery approaches, lack the precision needed to recover ground-truth graphs, even for small graphs and relatively large sample sizes. Furthermore, we identify the faithfulness property as a critical bottleneck: (i) it is likely to be violated across any reasonable dataset size range, and (ii) its violation directly undermines the performance of neural discovery methods. These findings lead us to conclude that progress within the current paradigm is fundamentally constrained, necessitating a paradigm shift in this domain.