A Meaningful Perturbation Metric for Evaluating Explainability Methods
This addresses the need for standardized evaluation metrics in explainable AI, offering a more reliable way to assess attribution methods for researchers and practitioners, though it is incremental in improving existing perturbation approaches.
The paper tackles the problem of evaluating attribution methods for deep neural networks by introducing a novel perturbation metric that uses image generation to inpaint high-relevance pixels, preserving image fidelity and achieving significantly higher correlation with human preferences compared to existing methods.
Deep neural networks (DNNs) have demonstrated remarkable success, yet their wide adoption is often hindered by their opaque decision-making. To address this, attribution methods have been proposed to assign relevance values to each part of the input. However, different methods often produce entirely different relevance maps, necessitating the development of standardized metrics to evaluate them. Typically, such evaluation is performed through perturbation, wherein high- or low-relevance regions of the input image are manipulated to examine the change in prediction. In this work, we introduce a novel approach, which harnesses image generation models to perform targeted perturbation. Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity. This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results. Through extensive experiments, we demonstrate the effectiveness of our approach in generating meaningful rankings across a wide range of models and attribution methods. Crucially, we establish that the ranking produced by our metric exhibits significantly higher correlation with human preferences compared to existing approaches, underscoring its potential for enhancing interpretability in DNNs.