Supervising Feature Influence
This addresses the reliability of causal influence analysis in machine learning, which is crucial for interpretability and bias detection, though it is incremental as it builds on existing causal testing methods.
The paper tackles the problem of covariate shift in causal testing, where standard training methods fail to constrain classifier behavior on atypical data points, leading to unreliable causal influence measures. The authors propose an active learning algorithm that constrains influence measures, proving theoretical guarantees and empirically showing it trains models with similar causal influences to labelers, better out-of-distribution generalization, and retained in-distribution accuracy.
Causal influence measures for machine learnt classifiers shed light on the reasons behind classification, and aid in identifying influential input features and revealing their biases. However, such analyses involve evaluating the classifier using datapoints that may be atypical of its training distribution. Standard methods for training classifiers that minimize empirical risk do not constrain the behavior of the classifier on such datapoints. As a result, training to minimize empirical risk does not distinguish among classifiers that agree on predictions in the training distribution but have wildly different causal influences. We term this problem covariate shift in causal testing and formally characterize conditions under which it arises. As a solution to this problem, we propose a novel active learning algorithm that constrains the influence measures of the trained model. We prove that any two predictors whose errors are close on both the original training distribution and the distribution of atypical points are guaranteed to have causal influences that are also close. Further, we empirically demonstrate with synthetic labelers that our algorithm trains models that (i) have similar causal influences as the labeler's model, and (ii) generalize better to out-of-distribution points while (iii) retaining their accuracy on in-distribution points.