AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions
This work addresses the challenge of improving decision quality in AI-assisted systems for practitioners and researchers, though it is incremental in clarifying existing concepts.
The paper tackles the problem of humans inappropriately relying on AI recommendations in decision-making, leading to poor outcomes, by formalizing the relationship between reliance and decision quality and showing conditions for effective human-AI complementarity.
In AI-assisted decision-making, a central promise of having a human-in-the-loop is that they should be able to complement the AI system by overriding its wrong recommendations. In practice, however, we often see that humans cannot assess the correctness of AI recommendations and, as a result, adhere to wrong or override correct advice. Different ways of relying on AI recommendations have immediate, yet distinct, implications for decision quality. Unfortunately, reliance and decision quality are often inappropriately conflated in the current literature on AI-assisted decision-making. In this work, we disentangle and formalize the relationship between reliance and decision quality, and we characterize the conditions under which human-AI complementarity is achievable. To illustrate how reliance and decision quality relate to one another, we propose a visual framework and demonstrate its usefulness for interpreting empirical findings, including the effects of interventions like explanations. Overall, our research highlights the importance of distinguishing between reliance behavior and decision quality in AI-assisted decision-making.