Causality-oriented robustness: exploiting general noise interventions
This work addresses the need for robust models in real-world applications like healthcare and single-cell analysis, offering a flexible approach that interpolates between in-distribution prediction and causality, though it appears incremental as it builds on existing causal frameworks.
The paper tackles the problem of developing prediction models robust to distribution shifts by proposing DRIG, a method that exploits general noise interventions to achieve robust predictions against unseen interventions, showing it protects against more diverse perturbations than existing methods.
Since distribution shifts are common in real-world applications, there is a pressing need to develop prediction models that are robust against such shifts. Existing frameworks, such as empirical risk minimization or distributionally robust optimization, either lack generalizability for unseen distributions or rely on postulated distance measures. Alternatively, causality offers a data-driven and structural perspective to robust predictions. However, the assumptions necessary for causal inference can be overly stringent, and the robustness offered by such causal models often lacks flexibility. In this paper, we focus on causality-oriented robustness and propose Distributional Robustness via Invariant Gradients (DRIG), a method that exploits general noise interventions in training data for robust predictions against unseen interventions, and naturally interpolates between in-distribution prediction and causality. In a linear setting, we prove that DRIG yields predictions that are robust among a data-dependent class of distribution shifts. Furthermore, we show that our framework includes anchor regression as a special case, and that it yields prediction models that protect against more diverse perturbations. We establish finite-sample results and extend our approach to semi-supervised domain adaptation to further improve prediction performance. Finally, we empirically validate our methods on synthetic simulations and on single-cell and intensive health care datasets.