Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers
This work addresses the need for interpretability in deep learning, particularly for models with product-type non-linearities, but it is incremental as it extends an existing framework to a specific layer type.
The paper tackled the problem of explaining neural network predictions by extending layer-wise relevance propagation to handle local renormalization layers, a common non-linearity in convolutional networks, and evaluated the method on CIFAR-10, Imagenet, and MIT Places datasets.
Layer-wise relevance propagation is a framework which allows to decompose the prediction of a deep neural network computed over a sample, e.g. an image, down to relevance scores for the single input dimensions of the sample such as subpixels of an image. While this approach can be applied directly to generalized linear mappings, product type non-linearities are not covered. This paper proposes an approach to extend layer-wise relevance propagation to neural networks with local renormalization layers, which is a very common product-type non-linearity in convolutional neural networks. We evaluate the proposed method for local renormalization layers on the CIFAR-10, Imagenet and MIT Places datasets.