Intelligent Reasoning Cues: A Framework and Case Study of the Roles of AI Information in Complex Decisions
This work addresses the gap in how AI interfaces affect reasoning processes for clinicians in high-stakes decisions, though it is incremental as it builds on existing theories.
The study tackled the problem of AI decision support systems failing to improve user decisions by proposing a framework of intelligent reasoning cues and testing it in a clinical sepsis treatment context, finding distinct influence patterns that inform design.
Artificial intelligence (AI)-based decision support systems can be highly accurate yet still fail to support users or improve decisions. Existing theories of AI-assisted decision-making focus on calibrating reliance on AI advice, leaving it unclear how different system designs might influence the reasoning processes underneath. We address this gap by reconsidering AI interfaces as collections of intelligent reasoning cues: discrete pieces of AI information that can individually influence decision-making. We then explore the roles of eight types of reasoning cues in a high-stakes clinical decision (treating patients with sepsis in intensive care). Through contextual inquiries with six teams and a think-aloud study with 25 physicians, we find that reasoning cues have distinct patterns of influence that can directly inform design. Our results also suggest that reasoning cues should prioritize tasks with high variability and discretion, adapt to ensure compatibility with evolving decision needs, and provide complementary, rigorous insights on complex cases.