Supporting Reflection and Forward-Looking Reasoning With Data-Driven Questions
For designers of human-AI decision-support systems, this work provides initial frameworks and tools to encourage reflection, though it is incremental and early-stage.
This paper addresses the problem of over-reliance on AI recommendations by promoting critical thinking through data-driven questions. It presents a question taxonomy, a medical prototype with clinician feedback, an LLM-based question generation method, and a cognitive engagement scale.
Many generative AI systems as well as decision-support systems (DSSs) provide operators with predictions or recommendations. Various studies show, however, that people can mistakenly adopt the erroneous results presented by those systems. Hence, it is crucial to promote critical thinking and reflection during interaction. One approach we are focusing on involves encouraging reflection during machine-assisted decision-making by presenting decision-makers with data-driven questions. In this short paper, we provide a brief overview of our work in that regard, namely: 1) the development of a question taxonomy, 2) the development of a prototype in the medical domain and the feedback received from clinicians, 3) a method for generating questions using a large language model, and 4) a proposed scale for measuring cognitive engagement in human-AI decision-making. In doing so, we contribute to the discussion about the design, development, and evaluation of tools for thought, i.e., AI systems that provoke critical thinking and enable novel ways of sense-making.