VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Network
This addresses the lack of clear interpretation mechanisms for GCNs, which is an incremental improvement for researchers and practitioners in computer vision.
The paper tackled the problem of interpreting Graph Convolutional Neural Networks (GCNs) in computer vision by proposing VS-CAM, a visualization method that generates semantic-aware heatmaps, resulting in highlighted regions that match objects more precisely than CNN-based CAM methods.
Graph convolutional neural network (GCN) has drawn increasing attention and attained good performance in various computer vision tasks, however, there lacks a clear interpretation of GCN's inner mechanism. For standard convolutional neural networks (CNNs), class activation mapping (CAM) methods are commonly used to visualize the connection between CNN's decision and image region by generating a heatmap. Nonetheless, such heatmap usually exhibits semantic-chaos when these CAMs are applied to GCN directly. In this paper, we proposed a novel visualization method particularly applicable to GCN, Vertex Semantic Class Activation Mapping (VS-CAM). VS-CAM includes two independent pipelines to produce a set of semantic-probe maps and a semantic-base map, respectively. Semantic-probe maps are used to detect the semantic information from semantic-base map to aggregate a semantic-aware heatmap. Qualitative results show that VS-CAM can obtain heatmaps where the highlighted regions match the objects much more precisely than CNN-based CAM. The quantitative evaluation further demonstrates the superiority of VS-CAM.