Understanding Individual Decisions of CNNs via Contrastive Backpropagation
This work addresses the need for better interpretability in deep learning for researchers and practitioners, though it is incremental as it builds on existing LRP methods.
The authors tackled the problem of non-discriminative saliency maps in understanding CNN decisions by proposing Contrastive Layer-wise Relevance Propagation (CLRP), which produced instance-specific, class-discriminative explanations, as shown through qualitative and quantitative evaluations like a Pointing Game and ablation study.
A number of backpropagation-based approaches such as DeConvNets, vanilla Gradient Visualization and Guided Backpropagation have been proposed to better understand individual decisions of deep convolutional neural networks. The saliency maps produced by them are proven to be non-discriminative. Recently, the Layer-wise Relevance Propagation (LRP) approach was proposed to explain the classification decisions of rectifier neural networks. In this work, we evaluate the discriminativeness of the generated explanations and analyze the theoretical foundation of LRP, i.e. Deep Taylor Decomposition. The experiments and analysis conclude that the explanations generated by LRP are not class-discriminative. Based on LRP, we propose Contrastive Layer-wise Relevance Propagation (CLRP), which is capable of producing instance-specific, class-discriminative, pixel-wise explanations. In the experiments, we use the CLRP to explain the decisions and understand the difference between neurons in individual classification decisions. We also evaluate the explanations quantitatively with a Pointing Game and an ablation study. Both qualitative and quantitative evaluations show that the CLRP generates better explanations than the LRP. The code is available.