IDGI: A Framework to Eliminate Explanation Noise from Integrated Gradients
This addresses interpretability issues in deep neural networks for researchers and practitioners, but it is incremental as it builds on existing IG-based methods.
The paper tackled the problem of noise in explanation saliency maps from Integrated Gradients methods, proposing the IDGI framework to reduce this noise and improve interpretability, with experiments showing drastic improvements on multiple metrics.
Integrated Gradients (IG) as well as its variants are well-known techniques for interpreting the decisions of deep neural networks. While IG-based approaches attain state-of-the-art performance, they often integrate noise into their explanation saliency maps, which reduce their interpretability. To minimize the noise, we examine the source of the noise analytically and propose a new approach to reduce the explanation noise based on our analytical findings. We propose the Important Direction Gradient Integration (IDGI) framework, which can be easily incorporated into any IG-based method that uses the Reimann Integration for integrated gradient computation. Extensive experiments with three IG-based methods show that IDGI improves them drastically on numerous interpretability metrics.