Friction in AI-Assisted Clinical Decision-Making: A Case Study on The Role of Questions and 'What-if' Scenarios
For human-AI interaction researchers, this work provides qualitative insights into designing friction to mitigate AI overreliance in clinical settings, though the findings are preliminary and based on a small sample.
The study investigated how data-driven questions and 'what-if' analysis are perceived by medical experts in AI-assisted clinical decision-making. Findings showed that questions were seen as unhelpful for reflection but useful as reminders, while 'what-if' scenarios were considered useful for improving patient care and as a training tool for novices.
Clinical decision-making is augmented by decision-support systems (DSSs). To counter overreliance on DSSs, several methods have been proposed that create friction in order to promote cognitive engagement and reflection. In this paper, we investigate how two such forms of friction, namely data-driven questions and `what-if' analysis, are perceived by medical experts. For a real-world decision task, we replicated a DSS used in clinical practice and gathered clinicians' feedback on a prototype through in-situ interviews (n=7). Our findings suggest that while the questions were perceived as unhelpful for reflective thinking, they could serve as reminders to consider relevant information. Furthermore, inspecting `what-if' hypotheticals was found useful for potentially improving patient care. Clinicians saw our prototype as a promising training tool for novice clinicians. From the clinicians' feedback, we make recommendations for designing friction in work practices. Our work contributes to human-AI interaction research, which aims to encourage reflection to mitigate AI overreliance.