Attributing AUC-ROC to Analyze Binary Classifier Performance
This work addresses the interpretability challenge in evaluating binary classifiers for researchers and practitioners, offering a novel attribution approach that is incremental over existing AUC-ROC analysis.
The paper tackles the problem of interpreting AUC-ROC scores for binary classifiers by developing techniques to attribute the metric to individual examples and pairs, enabling segmentation along interpretable dimensions. The result includes an efficient attribution method that reveals how inferior models can outperform superior ones on specific objectives, with experiments demonstrating its utility for analyzing decision boundaries and improvement potential.
Area Under the Receiver Operating Characteristic Curve (AUC-ROC) is a popular evaluation metric for binary classifiers. In this paper, we discuss techniques to segment the AUC-ROC along human-interpretable dimensions. AUC-ROC is not an additive/linear function over the data samples, therefore such segmenting the overall AUC-ROC is different from tabulating the AUC-ROC of data segments. To segment the overall AUC-ROC, we must first solve an \emph{attribution} problem to identify credit for individual examples. We observe that AUC-ROC, though non-linear over examples, is linear over \emph{pairs} of examples. This observation leads to a simple, efficient attribution technique for examples (example attributions), and for pairs of examples (pair attributions). We automatically slice these attributions using decision trees by making the tree predict the attributions; we use the notion of honest estimates along with a t-test to mitigate false discovery. Our experiments with the method show that an inferior model can outperform a superior model (trained to optimize a different training objective) on the inferior model's own training objective, a manifestation of Goodhart's Law. In contrast, AUC attributions enable a reasonable comparison. Example attributions can be used to slice this comparison. Pair attributions are used to categorize pairs of items -- one positively labeled and one negatively -- that the model has trouble separating. These categories identify the decision boundary of the classifier and the headroom to improve AUC.