COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification
This work addresses the need for better evaluation metrics in interpretability for researchers and practitioners, though it is incremental as it builds on existing saliency analysis.
The paper tackles the problem of evaluating the reliability of saliency methods in explaining deep learning model decisions for image classification, proposing the COSE metric to quantify consistency and sensitivity using data augmentations, and finds that most methods better explain transformer-based models while GradCAM performs best but has limitations.
We present a set of metrics that utilize vision priors to effectively assess the performance of saliency methods on image classification tasks. To understand behavior in deep learning models, many methods provide visual saliency maps emphasizing image regions that most contribute to a model prediction. However, there is limited work on analyzing the reliability of saliency methods in explaining model decisions. We propose the metric COnsistency-SEnsitivity (COSE) that quantifies the equivariant and invariant properties of visual model explanations using simple data augmentations. Through our metrics, we show that although saliency methods are thought to be architecture-independent, most methods could better explain transformer-based models over convolutional-based models. In addition, GradCAM was found to outperform other methods in terms of COSE but was shown to have limitations such as lack of variability for fine-grained datasets. The duality between consistency and sensitivity allow the analysis of saliency methods from different angles. Ultimately, we find that it is important to balance these two metrics for a saliency map to faithfully show model behavior.