A novel post-hoc explanation comparison metric and applications
This work addresses the need for standardized comparison metrics in explainable AI, which is crucial for researchers and practitioners evaluating model transparency, though it is incremental as it builds on existing explanation methods.
The paper tackles the problem of inconsistency in post-hoc explanation systems by introducing the Shreyan Distance, a metric to quantify differences between ranked feature importance lists, and applies it to compare SHAP and LIME, finding that average distances vary significantly between regression and classification tasks.
Explanatory systems make the behavior of machine learning models more transparent, but are often inconsistent. To quantify the differences between explanatory systems, this paper presents the Shreyan Distance, a novel metric based on the weighted difference between ranked feature importance lists produced by such systems. This paper uses the Shreyan Distance to compare two explanatory systems, SHAP and LIME, for both regression and classification learning tasks. Because we find that the average Shreyan Distance varies significantly between these two tasks, we conclude that consistency between explainers not only depends on inherent properties of the explainers themselves, but also the type of learning task. This paper further contributes the XAISuite library, which integrates the Shreyan distance algorithm into machine learning pipelines.