Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations
This addresses the need for systematic debugging of model errors to improve robustness and mitigate biases, though it builds incrementally on prior methods like counterfactual explanations and concept activation vectors.
The paper tackles the problem of understanding model mistakes by proposing conceptual counterfactual explanations (CCE), which explain errors in terms of human-understandable concepts, such as identifying spurious correlations from a single misclassified sample and generating clinician-confirmed insights in medical applications.
Understanding and explaining the mistakes made by trained models is critical to many machine learning objectives, such as improving robustness, addressing concept drift, and mitigating biases. However, this is often an ad hoc process that involves manually looking at the model's mistakes on many test samples and guessing at the underlying reasons for those incorrect predictions. In this paper, we propose a systematic approach, conceptual counterfactual explanations (CCE), that explains why a classifier makes a mistake on a particular test sample(s) in terms of human-understandable concepts (e.g. this zebra is misclassified as a dog because of faint stripes). We base CCE on two prior ideas: counterfactual explanations and concept activation vectors, and validate our approach on well-known pretrained models, showing that it explains the models' mistakes meaningfully. In addition, for new models trained on data with spurious correlations, CCE accurately identifies the spurious correlation as the cause of model mistakes from a single misclassified test sample. On two challenging medical applications, CCE generated useful insights, confirmed by clinicians, into biases and mistakes the model makes in real-world settings.