Explainer Divergence Scores (EDS): Some Post-Hoc Explanations May be Effective for Detecting Unknown Spurious Correlations
This addresses the challenge of reliably assessing explainer methods for AI interpretability, which is incremental as it improves evaluation frameworks rather than introducing a new detection technique.
The paper tackles the problem of evaluating post-hoc explainers for detecting spurious correlations in deep neural networks by proposing Explainer Divergence Scores (EDS), a new methodology that is interpretable and comparable, and finds that explainers often contain substantial information about spurious artifacts but in ways imperceptible to humans.
Recent work has suggested post-hoc explainers might be ineffective for detecting spurious correlations in Deep Neural Networks (DNNs). However, we show there are serious weaknesses with the existing evaluation frameworks for this setting. Previously proposed metrics are extremely difficult to interpret and are not directly comparable between explainer methods. To alleviate these constraints, we propose a new evaluation methodology, Explainer Divergence Scores (EDS), grounded in an information theory approach to evaluate explainers. EDS is easy to interpret and naturally comparable across explainers. We use our methodology to compare the detection performance of three different explainers - feature attribution methods, influential examples and concept extraction, on two different image datasets. We discover post-hoc explainers often contain substantial information about a DNN's dependence on spurious artifacts, but in ways often imperceptible to human users. This suggests the need for new techniques that can use this information to better detect a DNN's reliance on spurious correlations.