Guided Integrated Gradients: An Adaptive Path Method for Removing Noise
This addresses a specific issue in explainable AI for visual models, offering an incremental improvement over existing attribution methods.
The paper tackled the problem of noisy pixel attributions in Integrated Gradients for deep neural networks by proposing Guided Integrated Gradients, an adaptive path method that conditions the attribution path on the model, resulting in saliency maps better aligned with predictions and input images, with quantitative experiments showing it outperforms related methods.
Integrated Gradients (IG) is a commonly used feature attribution method for deep neural networks. While IG has many desirable properties, the method often produces spurious/noisy pixel attributions in regions that are not related to the predicted class when applied to visual models. While this has been previously noted, most existing solutions are aimed at addressing the symptoms by explicitly reducing the noise in the resulting attributions. In this work, we show that one of the causes of the problem is the accumulation of noise along the IG path. To minimize the effect of this source of noise, we propose adapting the attribution path itself -- conditioning the path not just on the image but also on the model being explained. We introduce Adaptive Path Methods (APMs) as a generalization of path methods, and Guided IG as a specific instance of an APM. Empirically, Guided IG creates saliency maps better aligned with the model's prediction and the input image that is being explained. We show through qualitative and quantitative experiments that Guided IG outperforms other, related methods in nearly every experiment.