BEExAI: Benchmark to Evaluate Explainable AI
This addresses the critical need for reliable evaluation of XAI methods, particularly as models become more complex, but it is incremental as it builds on existing metrics without introducing new paradigms.
The authors tackled the lack of a cohesive approach for evaluating post-hoc explainable AI methods by proposing BEExAI, a benchmark tool that enables large-scale comparison using selected evaluation metrics.
Recent research in explainability has given rise to numerous post-hoc attribution methods aimed at enhancing our comprehension of the outputs of black-box machine learning models. However, evaluating the quality of explanations lacks a cohesive approach and a consensus on the methodology for deriving quantitative metrics that gauge the efficacy of explainability post-hoc attribution methods. Furthermore, with the development of increasingly complex deep learning models for diverse data applications, the need for a reliable way of measuring the quality and correctness of explanations is becoming critical. We address this by proposing BEExAI, a benchmark tool that allows large-scale comparison of different post-hoc XAI methods, employing a set of selected evaluation metrics.