Semantic Interpretation and Validation of Graph Attention-based Explanations for GNN Models
This work addresses the need for reliable post-hoc semantic explanations in graph-based deep learning, particularly for applications like scene interpretation, though it appears incremental by extending existing attention-based methods.
The researchers tackled the problem of validating attention-based explanations for Graph Neural Networks by introducing semantically-informed perturbations and correlating attention weights with model accuracy, successfully identifying key semantic classes that enhance performance in a lidar pointcloud estimation model.
In this work, we propose a methodology for investigating the use of semantic attention to enhance the explainability of Graph Neural Network (GNN)-based models. Graph Deep Learning (GDL) has emerged as a promising field for tasks like scene interpretation, leveraging flexible graph structures to concisely describe complex features and relationships. As traditional explainability methods used in eXplainable AI (XAI) cannot be directly applied to such structures, graph-specific approaches are introduced. Attention has been previously employed to estimate the importance of input features in GDL, however, the fidelity of this method in generating accurate and consistent explanations has been questioned. To evaluate the validity of using attention weights as feature importance indicators, we introduce semantically-informed perturbations and correlate predicted attention weights with the accuracy of the model. Our work extends existing attention-based graph explainability methods by analysing the divergence in the attention distributions in relation to semantically sorted feature sets and the behaviour of a GNN model, efficiently estimating feature importance. We apply our methodology on a lidar pointcloud estimation model successfully identifying key semantic classes that contribute to enhanced performance, effectively generating reliable post-hoc semantic explanations.