Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes
This addresses the challenge of integrating AI into critical clinical decision-making for healthcare professionals, though it appears incremental in design approach.
The paper tackled the problem of clinical decision support tools (DSTs) failing in practice due to poor contextual fit by designing a new DST that automatically generates slides with embedded machine prognostics, and field evaluation showed clinicians were more likely to embrace it.
Clinical decision support tools (DST) promise improved healthcare outcomes by offering data-driven insights. While effective in lab settings, almost all DSTs have failed in practice. Empirical research diagnosed poor contextual fit as the cause. This paper describes the design and field evaluation of a radically new form of DST. It automatically generates slides for clinicians' decision meetings with subtly embedded machine prognostics. This design took inspiration from the notion of "Unremarkable Computing", that by augmenting the users' routines technology/AI can have significant importance for the users yet remain unobtrusive. Our field evaluation suggests clinicians are more likely to encounter and embrace such a DST. Drawing on their responses, we discuss the importance and intricacies of finding the right level of unremarkableness in DST design, and share lessons learned in prototyping critical AI systems as a situated experience.