On the Robustness of Removal-Based Feature Attributions
This work addresses the robustness of feature attribution methods for explaining machine learning predictions, which is crucial for trustworthy AI, but it is incremental as it extends prior focus from gradient-based to removal-based methods.
The paper tackles the problem of understanding the robustness of removal-based feature attribution methods to input and model perturbations, providing a unified theoretical analysis with derived upper bounds and empirical validation on synthetic and real-world data, showing that improving model Lipschitz regularity can increase attribution robustness.
To explain predictions made by complex machine learning models, many feature attribution methods have been developed that assign importance scores to input features. Some recent work challenges the robustness of these methods by showing that they are sensitive to input and model perturbations, while other work addresses this issue by proposing robust attribution methods. However, previous work on attribution robustness has focused primarily on gradient-based feature attributions, whereas the robustness of removal-based attribution methods is not currently well understood. To bridge this gap, we theoretically characterize the robustness properties of removal-based feature attributions. Specifically, we provide a unified analysis of such methods and derive upper bounds for the difference between intact and perturbed attributions, under settings of both input and model perturbations. Our empirical results on synthetic and real-world data validate our theoretical results and demonstrate their practical implications, including the ability to increase attribution robustness by improving the model's Lipschitz regularity.