Evaluation of Explanation Methods of AI -- CNNs in Image Classification Tasks with Reference-based and No-reference Metrics
This addresses the open research question of evaluating explanation quality for AI decisions in visual tasks, but it is incremental as it builds on existing metrics and applies them to known explainers.
The paper tackles the problem of evaluating post-hoc explainers for CNNs in image classification by generalizing methodologies using reference-based metrics (Pearson correlation, Similarity with ground truth Gaze Fixation Density Maps) and a no-reference stability metric, showing that the stability metric agrees with reference-based ones under image degradation and can be used when ground truth is unavailable.
The most popular methods in AI-machine learning paradigm are mainly black boxes. This is why explanation of AI decisions is of emergency. Although dedicated explanation tools have been massively developed, the evaluation of their quality remains an open research question. In this paper, we generalize the methodologies of evaluation of post-hoc explainers of CNNs' decisions in visual classification tasks with reference and no-reference based metrics. We apply them on our previously developed explainers (FEM, MLFEM), and popular Grad-CAM. The reference-based metrics are Pearson correlation coefficient and Similarity computed between the explanation map and its ground truth represented by a Gaze Fixation Density Map obtained with a psycho-visual experiment. As a no-reference metric, we use stability metric, proposed by Alvarez-Melis and Jaakkola. We study its behaviour, consensus with reference-based metrics and show that in case of several kinds of degradation on input images, this metric is in agreement with reference-based ones. Therefore, it can be used for evaluation of the quality of explainers when the ground truth is not available.