Robustness and Usefulness in AI Explanation Methods
This work highlights critical limitations in widely adopted explanation methods, which is important for practitioners and regulators relying on AI transparency.
The paper evaluated three popular AI explanation methods (LIME, SmoothGrad, SHAP) for robustness, understandability, and usability, concluding that they are insufficient and potentially harmful compared to not using them.
Explainability in machine learning has become incredibly important as machine learning-powered systems become ubiquitous and both regulation and public sentiment begin to demand an understanding of how these systems make decisions. As a result, a number of explanation methods have begun to receive widespread adoption. This work summarizes, compares, and contrasts three popular explanation methods: LIME, SmoothGrad, and SHAP. We evaluate these methods with respect to: robustness, in the sense of sample complexity and stability; understandability, in the sense that provided explanations are consistent with user expectations; and usability, in the sense that the explanations allow for the model to be modified based on the output. This work concludes that current explanation methods are insufficient; that putting faith in and adopting these methods may actually be worse than simply not using them.