Explainable AI And Visual Reasoning: Insights From Radiology
This addresses the problem of low trust and adoption of XAI in radiology, offering incremental insights for designing more human-centered explanations.
The paper investigates why explainable AI (XAI) explanations in radiology fail to gain human trust, attributing it to a lack of intuitive coverage of evidence, and proposes that explanations mirroring human reasoning processes could improve trust and adoption.
Why do explainable AI (XAI) explanations in radiology, despite their promise of transparency, still fail to gain human trust? Current XAI approaches provide justification for predictions, however, these do not meet practitioners' needs. These XAI explanations lack intuitive coverage of the evidentiary basis for a given classification, posing a significant barrier to adoption. We posit that XAI explanations that mirror human processes of reasoning and justification with evidence may be more useful and trustworthy than traditional visual explanations like heat maps. Using a radiology case study, we demonstrate how radiology practitioners get other practitioners to see a diagnostic conclusion's validity. Machine-learned classifications lack this evidentiary grounding and consequently fail to elicit trust and adoption by potential users. Insights from this study may generalize to guiding principles for human-centered explanation design based on human reasoning and justification of evidence.