Prediction-Powered Causal Inferences
This addresses the bottleneck of annotation costs for researchers in scientific fields, offering a novel solution for causal inference in unlabeled experiments.
The paper tackles the problem of high data annotation costs in scientific experiments by proposing Prediction-Powered Causal Inferences (PPCI) to estimate treatment effects using unlabeled target data, achieving valid causal inference on complex experiments without human annotations for the first time.
In many scientific experiments, the data annotating cost constraints the pace for testing novel hypotheses. Yet, modern machine learning pipelines offer a promising solution, provided their predictions yield correct conclusions. We focus on Prediction-Powered Causal Inferences (PPCI), i.e., estimating the treatment effect in an unlabeled target experiment, relying on training data with the same outcome annotated but potentially different treatment or effect modifiers. We first show that conditional calibration guarantees valid PPCI at population level. Then, we introduce a sufficient representation constraint transferring validity across experiments, which we propose to enforce in practice in Deconfounded Empirical Risk Minimization, our new model-agnostic training objective. We validate our method on synthetic and real-world scientific data, solving impossible problem instances for Empirical Risk Minimization even with standard invariance constraints. In particular, for the first time, we achieve valid causal inference on a scientific experiment with complex recording and no human annotations, fine-tuning a foundational model on our similar annotated experiment.