The XAISuite framework and the implications of explanatory system dissonance
It addresses the problem of explanatory system dissonance for researchers and practitioners in interpretable machine learning, though it is incremental as it builds on existing methods.
This paper tackles the inconsistency between explanatory systems like SHAP and LIME by comparing their importance scores across 14 models and 4 datasets, finding that importance magnitude does not significantly affect consistency and that similarity in scores cannot predict model accuracy.
Explanatory systems make machine learning models more transparent. However, they are often inconsistent. In order to quantify and isolate possible scenarios leading to this discrepancy, this paper compares two explanatory systems, SHAP and LIME, based on the correlation of their respective importance scores using 14 machine learning models (7 regression and 7 classification) and 4 tabular datasets (2 regression and 2 classification). We make two novel findings. Firstly, the magnitude of importance is not significant in explanation consistency. The correlations between SHAP and LIME importance scores for the most important features may or may not be more variable than the correlation between SHAP and LIME importance scores averaged across all features. Secondly, the similarity between SHAP and LIME importance scores cannot predict model accuracy. In the process of our research, we construct an open-source library, XAISuite, that unifies the process of training and explaining models. Finally, this paper contributes a generalized framework to better explain machine learning models and optimize their performance.