Superclass-Guided Representation Disentanglement for Spurious Correlation Mitigation
This addresses domain generalization challenges for machine learning practitioners by reducing reliance on impractical annotations, though it is incremental as it builds on prior work on spurious correlation mitigation.
The paper tackles the problem of group robustness to spurious correlations in domain generalization by proposing a method that uses superclass information to disentangle features, eliminating the need for auxiliary annotations. It significantly outperforms baselines across diverse datasets, as shown in quantitative metrics and qualitative visualizations.
To enhance group robustness to spurious correlations, prior work often relies on auxiliary annotations for groups or spurious features and assumes identical sets of groups across source and target domains. These two requirements are both unnatural and impractical in real-world settings. To overcome these limitations, we propose a method that leverages the semantic structure inherent in class labels--specifically, superclass information--to naturally reduce reliance on spurious features. Our model employs gradient-based attention guided by a pre-trained vision-language model to disentangle superclass-relevant and irrelevant features. Then, by promoting the use of all superclass-relevant features for prediction, our approach achieves robustness to more complex spurious correlations without the need to annotate any source samples. Experiments across diverse datasets demonstrate that our method significantly outperforms baselines in domain generalization tasks, with clear improvements in both quantitative metrics and qualitative visualizations.