Explainability is NOT a Game
This work critiques a foundational tool in explainable AI, potentially impacting its use in critical domains like healthcare or finance.
The paper argues that Shapley values, a common method for feature importance in explainable AI, can be misleading by overemphasizing irrelevant features and underemphasizing relevant ones, challenging their reliability in high-stakes applications.
Explainable artificial intelligence (XAI) aims to help human decision-makers in understanding complex machine learning (ML) models. One of the hallmarks of XAI are measures of relative feature importance, which are theoretically justified through the use of Shapley values. This paper builds on recent work and offers a simple argument for why Shapley values can provide misleading measures of relative feature importance, by assigning more importance to features that are irrelevant for a prediction, and assigning less importance to features that are relevant for a prediction. The significance of these results is that they effectively challenge the many proposed uses of measures of relative feature importance in a fast-growing range of high-stakes application domains.