The Blame Problem in Evaluating Local Explanations, and How to Tackle it
This addresses a critical issue for researchers and practitioners in explainable AI by highlighting flaws in current evaluation methods, though it is incremental as it builds on existing taxonomies without proposing a new solution.
The study identifies the 'blame problem' in evaluating local model-agnostic explanation techniques, where most evaluation methods except those using ground truth from interpretable models are flawed, and argues this category is more reasonable but still has limitations, concluding that evaluation remains an open problem.
The number of local model-agnostic explanation techniques proposed has grown rapidly recently. One main reason is that the bar for developing new explainability techniques is low due to the lack of optimal evaluation measures. Without rigorous measures, it is hard to have concrete evidence of whether the new explanation techniques can significantly outperform their predecessors. Our study proposes a new taxonomy for evaluating local explanations: robustness, evaluation using ground truth from synthetic datasets and interpretable models, model randomization, and human-grounded evaluation. Using this proposed taxonomy, we highlight that all categories of evaluation methods, except those based on the ground truth from interpretable models, suffer from a problem we call the "blame problem." In our study, we argue that this category of evaluation measure is a more reasonable method for evaluating local model-agnostic explanations. However, we show that even this category of evaluation measures has further limitations. The evaluation of local explanations remains an open research problem.