Direct-Effect Risk Minimization for Domain Generalization
This addresses reliability concerns in machine learning for domain generalization, particularly for applications where spurious correlations vary across domains, though it appears incremental as it builds on existing methods.
The authors tackled the problem of out-of-distribution generalization under correlation shift by introducing direct and indirect effects from causal inference, proposing a two-stage algorithm to remove indirect effects and improve generalization performance, with experiments on 5 correlation-shifted datasets and the DomainBed benchmark verifying effectiveness.
We study the problem of out-of-distribution (o.o.d.) generalization where spurious correlations of attributes vary across training and test domains. This is known as the problem of correlation shift and has posed concerns on the reliability of machine learning. In this work, we introduce the concepts of direct and indirect effects from causal inference to the domain generalization problem. We argue that models that learn direct effects minimize the worst-case risk across correlation-shifted domains. To eliminate the indirect effects, our algorithm consists of two stages: in the first stage, we learn an indirect-effect representation by minimizing the prediction error of domain labels using the representation and the class labels; in the second stage, we remove the indirect effects learned in the first stage by matching each data with another data of similar indirect-effect representation but of different class labels in the training and validation phase. Our approach is shown to be compatible with existing methods and improve the generalization performance of them on correlation-shifted datasets. Experiments on 5 correlation-shifted datasets and the DomainBed benchmark verify the effectiveness of our approach.