ViT-CX: Causal Explanation of Vision Transformers
This addresses the need for better explainable AI methods specifically for ViTs, which is incremental as it builds on existing XAI approaches but targets a specific model type.
The paper tackles the problem of explaining Vision Transformers (ViTs) by proposing ViT-CX, a novel method based on patch embeddings and causal impacts, which produces more meaningful saliency maps and better reveals important evidence for predictions compared to previous methods, with significantly improved faithfulness to the model.
Despite the popularity of Vision Transformers (ViTs) and eXplainable AI (XAI), only a few explanation methods have been designed specially for ViTs thus far. They mostly use attention weights of the [CLS] token on patch embeddings and often produce unsatisfactory saliency maps. This paper proposes a novel method for explaining ViTs called ViT-CX. It is based on patch embeddings, rather than attentions paid to them, and their causal impacts on the model output. Other characteristics of ViTs such as causal overdetermination are also considered in the design of ViT-CX. The empirical results show that ViT-CX produces more meaningful saliency maps and does a better job revealing all important evidence for the predictions than previous methods. The explanation generated by ViT-CX also shows significantly better faithfulness to the model. The codes and appendix are available at https://github.com/vaynexie/CausalX-ViT.