On the Faithfulness of Vision Transformer Explanations
This work addresses the critical need for reliable evaluation metrics in Vision Transformer explainability, which is incremental but important for improving interpretability in AI systems.
The paper tackles the problem of evaluating the faithfulness of Vision Transformer explanations by introducing the Salience-guided Faithfulness Coefficient (SaCo), a novel metric that reliably measures how well salience scores reflect true model rationales, showing it outperforms existing metrics and identifies methods like gradient and multi-layer aggregation to enhance faithfulness.
To interpret Vision Transformers, post-hoc explanations assign salience scores to input pixels, providing human-understandable heatmaps. However, whether these interpretations reflect true rationales behind the model's output is still underexplored. To address this gap, we study the faithfulness criterion of explanations: the assigned salience scores should represent the influence of the corresponding input pixels on the model's predictions. To evaluate faithfulness, we introduce Salience-guided Faithfulness Coefficient (SaCo), a novel evaluation metric leveraging essential information of salience distribution. Specifically, we conduct pair-wise comparisons among distinct pixel groups and then aggregate the differences in their salience scores, resulting in a coefficient that indicates the explanation's degree of faithfulness. Our explorations reveal that current metrics struggle to differentiate between advanced explanation methods and Random Attribution, thereby failing to capture the faithfulness property. In contrast, our proposed SaCo offers a reliable faithfulness measurement, establishing a robust metric for interpretations. Furthermore, our SaCo demonstrates that the use of gradient and multi-layer aggregation can markedly enhance the faithfulness of attention-based explanation, shedding light on potential paths for advancing Vision Transformer explainability.