An Objective Metric for Explainable AI: How and Why to Estimate the Degree of Explainability
This addresses the challenge of quantifying explainability for users in fields like healthcare and finance, though it is incremental as it builds on existing theoretical models.
The paper tackles the problem of objectively evaluating explainability in AI by proposing a new model-agnostic metric to measure the Degree of Explainability, with results from experiments involving over 190 participants showing statistically significant alignment with expectations (P values < .01).
Explainable AI was born as a pathway to allow humans to explore and understand the inner working of complex systems. However, establishing what is an explanation and objectively evaluating explainability are not trivial tasks. This paper presents a new model-agnostic metric to measure the Degree of Explainability of information in an objective way. We exploit a specific theoretical model from Ordinary Language Philosophy called the Achinstein's Theory of Explanations, implemented with an algorithm relying on deep language models for knowledge graph extraction and information retrieval. To understand whether this metric can measure explainability, we devised a few experiments and user studies involving more than 190 participants, evaluating two realistic systems for healthcare and finance using famous AI technology, including Artificial Neural Networks and TreeSHAP. The results we obtained are statistically significant (with P values lower than .01), suggesting that our proposed metric for measuring the Degree of Explainability is robust in several scenarios, and it aligns with concrete expectations.