Faithful Attention Explainer: Verbalizing Decisions Based on Discriminative Features
This work addresses the need for interpretable AI explanations for users, though it is incremental as it builds on existing attention-based methods.
The paper tackles the problem of generating faithful textual explanations for model decisions by proposing the Faithful Attention Explainer (FAE) framework, which uses an attention module to link visual features to words and achieves promising performance in caption quality and decision-relevance metrics on CUB and ACT-X datasets.
In recent years, model explanation methods have been designed to interpret model decisions faithfully and intuitively so that users can easily understand them. In this paper, we propose a framework, Faithful Attention Explainer (FAE), capable of generating faithful textual explanations regarding the attended-to features. Towards this goal, we deploy an attention module that takes the visual feature maps from the classifier for sentence generation. Furthermore, our method successfully learns the association between features and words, which allows a novel attention enforcement module for attention explanation. Our model achieves promising performance in caption quality metrics and a faithful decision-relevance metric on two datasets (CUB and ACT-X). In addition, we show that FAE can interpret gaze-based human attention, as human gaze indicates the discriminative features that humans use for decision-making, demonstrating the potential of deploying human gaze for advanced human-AI interaction.