A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME
This work highlights critical limitations in widely used XAI tools, cautioning researchers and practitioners in fields like biomedicine about potential misinterpretations, though it is incremental as it builds on existing methods.
The authors analyzed SHAP and LIME explainable AI methods, finding that their outputs are significantly influenced by the machine learning model used and feature collinearity, as demonstrated in a biomedical case study on myocardial infarction classification.
eXplainable artificial intelligence (XAI) methods have emerged to convert the black box of machine learning (ML) models into a more digestible form. These methods help to communicate how the model works with the aim of making ML models more transparent and increasing the trust of end-users into their output. SHapley Additive exPlanations (SHAP) and Local Interpretable Model Agnostic Explanation (LIME) are two widely used XAI methods, particularly with tabular data. In this perspective piece, we discuss the way the explainability metrics of these two methods are generated and propose a framework for interpretation of their outputs, highlighting their weaknesses and strengths. Specifically, we discuss their outcomes in terms of model-dependency and in the presence of collinearity among the features, relying on a case study from the biomedical domain (classification of individuals with or without myocardial infarction). The results indicate that SHAP and LIME are highly affected by the adopted ML model and feature collinearity, raising a note of caution on their usage and interpretation.