Counterfactual (Non-)identifiability of Learned Structural Causal Models
This work warns practitioners in causal inference and machine learning about fundamental limitations in using deep generative models for counterfactual predictions, highlighting the need for parametric assumptions to avoid unreliable results.
The paper tackles the problem of non-identifiability in counterfactual inference using learned Structural Causal Models from observational data, proving identifiability only under monotonic generation mechanisms with single-dimensional exogenous variables and providing an impossibility result for general cases, while proposing a method to estimate worst-case errors that shows negligible errors for identifiable models and informative bounds for non-identifiable ones.
Recent advances in probabilistic generative modeling have motivated learning Structural Causal Models (SCM) from observational datasets using deep conditional generative models, also known as Deep Structural Causal Models (DSCM). If successful, DSCMs can be utilized for causal estimation tasks, e.g., for answering counterfactual queries. In this work, we warn practitioners about non-identifiability of counterfactual inference from observational data, even in the absence of unobserved confounding and assuming known causal structure. We prove counterfactual identifiability of monotonic generation mechanisms with single dimensional exogenous variables. For general generation mechanisms with multi-dimensional exogenous variables, we provide an impossibility result for counterfactual identifiability, motivating the need for parametric assumptions. As a practical approach, we propose a method for estimating worst-case errors of learned DSCMs' counterfactual predictions. The size of this error can be an essential metric for deciding whether or not DSCMs are a viable approach for counterfactual inference in a specific problem setting. In evaluation, our method confirms negligible counterfactual errors for an identifiable SCM from prior work, and also provides informative error bounds on counterfactual errors for a non-identifiable synthetic SCM.