2-Factor Retrieval for Improved Human-AI Decision Making in Radiology
This addresses the need for verifiable AI assistance in clinical decision support systems, though it is incremental as it builds on prior work in explainable AI.
The study tackled the problem of improving human-AI decision-making in radiology by comparing existing explainable AI techniques with a new '2-factor retrieval' method, which increased clinician accuracy in chest X-ray diagnoses, especially for radiologists with low confidence.
Human-machine teaming in medical AI requires us to understand to what degree a trained clinician should weigh AI predictions. While previous work has shown the potential of AI assistance at improving clinical predictions, existing clinical decision support systems either provide no explainability of their predictions or use techniques like saliency and Shapley values, which do not allow for physician-based verification. To address this gap, this study compares previously used explainable AI techniques with a newly proposed technique termed '2-factor retrieval (2FR)', which is a combination of interface design and search retrieval that returns similarly labeled data without processing this data. This results in a 2-factor security blanket where: (a) correct images need to be retrieved by the AI; and (b) humans should associate the retrieved images with the current pathology under test. We find that when tested on chest X-ray diagnoses, 2FR leads to increases in clinician accuracy, with particular improvements when clinicians are radiologists and have low confidence in their decision. Our results highlight the importance of understanding how different modes of human-AI decision making may impact clinician accuracy in clinical decision support systems.