Causal Effect Regularization: Automated Detection and Removal of Spurious Attributes
This addresses the issue of poor generalization in machine learning models due to spurious correlations, which is critical for real-world applications where such attributes are unknown, though it builds incrementally on prior work by automating detection.
The paper tackles the problem of classifiers relying on spurious attributes in datasets, which harms generalization under correlation shifts, by proposing a method to automatically detect and remove these attributes using causal effect estimation and regularization. The result shows significant improvement in reducing dependence on spurious attributes (measured by ΔProb) while maintaining or improving accuracy across synthetic and real-world datasets.
In many classification datasets, the task labels are spuriously correlated with some input attributes. Classifiers trained on such datasets often rely on these attributes for prediction, especially when the spurious correlation is high, and thus fail to generalize whenever there is a shift in the attributes' correlation at deployment. If we assume that the spurious attributes are known a priori, several methods have been proposed to learn a classifier that is invariant to the specified attributes. However, in real-world data, information about spurious attributes is typically unavailable. Therefore, we propose a method to automatically identify spurious attributes by estimating their causal effect on the label and then use a regularization objective to mitigate the classifier's reliance on them. Compared to a recent method for identifying spurious attributes, we find that our method is more accurate in removing the attribute from the learned model, especially when spurious correlation is high. Specifically, across synthetic, semi-synthetic, and real-world datasets, our method shows significant improvement in a metric used to quantify the dependence of a classifier on spurious attributes ($Δ$Prob), while obtaining better or similar accuracy. In addition, our method mitigates the reliance on spurious attributes even under noisy estimation of causal effects. To explain the empirical robustness of our method, we create a simple linear classification task with two sets of attributes: causal and spurious. We prove that our method only requires that the ranking of estimated causal effects is correct across attributes to select the correct classifier.