Shaping Up SHAP: Enhancing Stability through Layer-Wise Neighbor Selection
This addresses the issue of unreliable explanations for black-box models, which is critical for fairness and transparency in decision-making, though it appears incremental as it builds directly on Kernel SHAP.
The paper tackled the instability problem in Kernel SHAP, a popular explainability method, by modifying its neighbor selection procedure to achieve full stability without losing fidelity, resulting in a novel method that is stable, efficient, and meaningful.
Machine learning techniques, such as deep learning and ensemble methods, are widely used in various domains due to their ability to handle complex real-world tasks. However, their black-box nature has raised multiple concerns about the fairness, trustworthiness, and transparency of computer-assisted decision-making. This has led to the emergence of local post-hoc explainability methods, which offer explanations for individual decisions made by black-box algorithms. Among these methods, Kernel SHAP is widely used due to its model-agnostic nature and its well-founded theoretical framework. Despite these strengths, Kernel SHAP suffers from high instability: different executions of the method with the same inputs can lead to significantly different explanations, which diminishes the relevance of the explanations. The contribution of this paper is two-fold. On the one hand, we show that Kernel SHAP's instability is caused by its stochastic neighbor selection procedure, which we adapt to achieve full stability without compromising explanation fidelity. On the other hand, we show that by restricting the neighbors generation to perturbations of size 1 -- which we call the coalitions of Layer 1 -- we obtain a novel feature-attribution method that is fully stable, computationally efficient, and still meaningful.