Towards Rigorous Explainability by Feature Attribution
For practitioners and regulators needing trustworthy explanations in high-stakes ML, this work highlights the need for rigorous alternatives to popular but flawed non-symbolic methods.
This paper argues that non-symbolic explanation methods like SHAP lack rigor and can mislead decision-makers, especially in high-stakes ML. It overviews ongoing efforts to replace them with rigorous symbolic methods for feature attribution.
For around a decade, non-symbolic methods have been the option of choice when explaining complex machine learning (ML) models. Unfortunately, such methods lack rigor and can mislead human decision-makers. In high-stakes uses of ML, the lack of rigor is especially problematic. One prime example of provable lack of rigor is the adoption of Shapley values in explainable artificial intelligence (XAI), with the tool SHAP being a ubiquitous example. This paper overviews the ongoing efforts towards using rigorous symbolic methods of XAI as an alternative to non-rigorous non-symbolic approaches, concretely for assigning relative feature importance.