A Survey on Human-Centered Evaluation of Explainable AI Methods in Clinical Decision Support Systems
This addresses the inconsistent evaluation of XAI in clinical settings, which is crucial for adoption, but it is incremental as it synthesizes existing studies rather than introducing new methods.
The study conducted a systematic survey of 31 human-centered evaluations of explainable AI methods in clinical decision support systems, finding that over 80% use post-hoc approaches like SHAP and Grad-CAM with small clinician samples, and that explanations improve trust but increase cognitive load and misalign with domain reasoning.
Explainable Artificial Intelligence (XAI) is essential for the transparency and clinical adoption of Clinical Decision Support Systems (CDSS). However, the real-world effectiveness of existing XAI methods remains limited and is inconsistently evaluated. This study conducts a systematic PRISMA-guided survey of 31 human-centered evaluations (HCE) of XAI applied to CDSS, classifying them by XAI methodology, evaluation design, and adoption barrier. Our findings reveal that most existing studies employ post-hoc, model-agnostic approaches such as SHAP and Grad-CAM, typically assessed through small-scale clinician studies. The results show that over 80% of the studies adopt post-hoc, model-agnostic approaches such as SHAP and Grad-CAM, and that clinician sample sizes remain below 25 participants. The findings indicate that explanations generally improve clinician trust and diagnostic confidence, but frequently increase cognitive load and exhibit misalignment with domain reasoning processes. To bridge these gaps, we propose a stakeholder-centric evaluation framework that integrates socio-technical principles and human-computer interaction to guide the future development of clinically viable and trustworthy XAI-based CDSS.