Post-hoc Self-explanation of CNNs
This work addresses the interpretability of CNNs for users needing transparent AI decisions, but it is incremental as it builds on existing self-explainable models with a modified classifier.
The paper tackled the problem of standard CNNs lacking accurate built-in prototypes for self-explanation by replacing the final linear layer with a k-means-based classifier, which maintained performance while enabling post-hoc explanations. Empirical results with ResNet34 showed that using shallower feature activations (e.g., from blocks B234) led to a trade-off between improved semantic fidelity and a slight reduction in predictive performance.
Although standard Convolutional Neural Networks (CNNs) can be mathematically reinterpreted as Self-Explainable Models (SEMs), their built-in prototypes do not on their own accurately represent the data. Replacing the final linear layer with a $k$-means-based classifier addresses this limitation without compromising performance. This work introduces a common formalization of $k$-means-based post-hoc explanations for the classifier, the encoder's final output (B4), and combinations of intermediate feature activations. The latter approach leverages the spatial consistency of convolutional receptive fields to generate concept-based explanation maps, which are supported by gradient-free feature attribution maps. Empirical evaluation with a ResNet34 shows that using shallower, less compressed feature activations, such as those from the last three blocks (B234), results in a trade-off between semantic fidelity and a slight reduction in predictive performance.