A-FMI: Learning Attributions from Deep Networks via Feature Map Importance
This work improves interpretability for CNN users by providing more faithful attributions, though it is incremental as it builds on existing gradient-based methods.
The paper tackled the problem of gradient saturation and feature redundancy in gradient-based attribution methods for CNNs, proposing A-FMI to address these issues and showing superior explanation performance on ImageNet compared to existing methods.
Gradient-based attribution methods can aid in the understanding of convolutional neural networks (CNNs). However, the redundancy of attribution features and the gradient saturation problem, which weaken the ability to identify significant features and cause an explanation focus shift, are challenges that attribution methods still face. In this work, we propose: 1) an essential characteristic, Strong Relevance, when selecting attribution features; 2) a new concept, feature map importance (FMI), to refine the contribution of each feature map, which is faithful to the CNN model; and 3) a novel attribution method via FMI, termed A-FMI, to address the gradient saturation problem, which couples the target image with a reference image, and assigns the FMI to the difference-from-reference at the granularity of feature map. Through visual inspections and qualitative evaluations on the ImageNet dataset, we show the compelling advantages of A-FMI on its faithfulness, insensitivity to the choice of reference, class discriminability, and superior explanation performance compared with popular attribution methods across varying CNN architectures.