Revealing Similar Semantics Inside CNNs: An Interpretable Concept-based Comparison of Feature Spaces
This work addresses the need for transparency in AI for safety-critical applications by enabling semantic comparison of CNNs, though it is incremental as it builds on existing explainable AI techniques.
The paper tackles the problem of interpreting what convolutional neural networks (CNNs) have learned by proposing two methods to estimate layer-wise similarity of semantic information in latent spaces, revealing that similar semantic concepts are learned across different CNN architectures and emerge at similar relative depths.
Safety-critical applications require transparency in artificial intelligence (AI) components, but widely used convolutional neural networks (CNNs) widely used for perception tasks lack inherent interpretability. Hence, insights into what CNNs have learned are primarily based on performance metrics, because these allow, e.g., for cross-architecture CNN comparison. However, these neglect how knowledge is stored inside. To tackle this yet unsolved problem, our work proposes two methods for estimating the layer-wise similarity between semantic information inside CNN latent spaces. These allow insights into both the flow and likeness of semantic information within CNN layers, and into the degree of their similarity between different network architectures. As a basis, we use two renowned explainable artificial intelligence (XAI) techniques, which are used to obtain concept activation vectors, i.e., global vector representations in the latent space. These are compared with respect to their activation on test inputs. When applied to three diverse object detectors and two datasets, our methods reveal that (1) similar semantic concepts are learned regardless of the CNN architecture, and (2) similar concepts emerge in similar relative layer depth, independent of the total number of layers. Finally, our approach poses a promising step towards semantic model comparability and comprehension of how different CNNs process semantic information.