Valid P-Value for Deep Learning-Driven Salient Region
This provides a statistical reliability measure for saliency maps in deep learning, addressing a key bottleneck in model interpretability, though it is incremental as it builds on existing selective inference frameworks.
The authors tackled the problem of quantifying the reliability of saliency maps in deep learning by proposing a method to compute p-values for salient regions, which provably controls false positive detections, as demonstrated on synthetic and real datasets.
Various saliency map methods have been proposed to interpret and explain predictions of deep learning models. Saliency maps allow us to interpret which parts of the input signals have a strong influence on the prediction results. However, since a saliency map is obtained by complex computations in deep learning models, it is often difficult to know how reliable the saliency map itself is. In this study, we propose a method to quantify the reliability of a salient region in the form of p-values. Our idea is to consider a salient region as a selected hypothesis by the trained deep learning model and employ the selective inference framework. The proposed method can provably control the probability of false positive detections of salient regions. We demonstrate the validity of the proposed method through numerical examples in synthetic and real datasets. Furthermore, we develop a Keras-based framework for conducting the proposed selective inference for a wide class of CNNs without additional implementation cost.