Unsupervised Learning of Neural Networks to Explain Neural Networks (extended abstract)
This addresses the need for better interpretability in deep learning for researchers and practitioners, though it is incremental as it builds on existing auto-encoder and knowledge distillation techniques.
The paper tackles the problem of interpreting features in convolutional neural networks (CNNs) by proposing an unsupervised method to learn an explainer network that decomposes feature maps into object-part features, significantly boosting interpretability without harming discrimination power.
This paper presents an unsupervised method to learn a neural network, namely an explainer, to interpret a pre-trained convolutional neural network (CNN), i.e., the explainer uses interpretable visual concepts to explain features in middle conv-layers of a CNN. Given feature maps of a conv-layer of the CNN, the explainer performs like an auto-encoder, which decomposes the feature maps into object-part features. The object-part features are learned to reconstruct CNN features without much loss of information. We can consider the disentangled representations of object parts a paraphrase of CNN features, which help people understand the knowledge encoded by the CNN. More crucially, we learn the explainer via knowledge distillation without using any annotations of object parts or textures for supervision. In experiments, our method was widely used to interpret features of different benchmark CNNs, and explainers significantly boosted the feature interpretability without hurting the discrimination power of the CNNs.