Standardizing Structural Causal Models
This addresses a methodological issue for researchers benchmarking causal inference algorithms, potentially improving generalization to real-world settings, though it is incremental as it builds on existing SCM frameworks.
The paper tackles the problem of synthetic datasets from structural causal models (SCMs) having artifacts like increasing variances and correlations along causal orderings, which can bias benchmarking of causal structure learning algorithms. They propose internally-standardized SCMs (iSCMs) that eliminate these artifacts by construction, making them not Var-sortable and mostly not R²-sortable for common graph families.
Synthetic datasets generated by structural causal models (SCMs) are commonly used for benchmarking causal structure learning algorithms. However, the variances and pairwise correlations in SCM data tend to increase along the causal ordering. Several popular algorithms exploit these artifacts, possibly leading to conclusions that do not generalize to real-world settings. Existing metrics like $\operatorname{Var}$-sortability and $\operatorname{R^2}$-sortability quantify these patterns, but they do not provide tools to remedy them. To address this, we propose internally-standardized structural causal models (iSCMs), a modification of SCMs that introduces a standardization operation at each variable during the generative process. By construction, iSCMs are not $\operatorname{Var}$-sortable. We also find empirical evidence that they are mostly not $\operatorname{R^2}$-sortable for commonly-used graph families. Moreover, contrary to the post-hoc standardization of data generated by standard SCMs, we prove that linear iSCMs are less identifiable from prior knowledge on the weights and do not collapse to deterministic relationships in large systems, which may make iSCMs a useful model in causal inference beyond the benchmarking problem studied here. Our code is publicly available at: https://github.com/werkaaa/iscm.