Raising the Stakes: Assessing the Influence of Stakes on User Reliance Behavior in Human-AI Decision-Making
For researchers and designers of human-AI systems, this work highlights the critical but often overlooked impact of stakes on reliance behavior, showing that increased effort does not guarantee better calibration.
The paper investigates how perceived stakes affect user reliance on AI in decision-making, finding that higher stakes lead to longer deliberation but less calibrated reliance, with increased deference to incorrect AI advice over time.
Human-AI collaboration is often proposed to improve high-stakes decision-making, yet the influence of increased stakes and imperfect AI on decision-making strategies is not fully understood. Studying such behavior in realistic settings is challenging, as application-grounded evaluations are costly, rely on experts, or lack meaningful consequences for decision errors. To address this, we introduce Blockies, a parametric dataset generator for visual diagnostic tasks, and conduct an empirical study examining how perceived stakes influence reliance calibration and behavior. Results show that raised stakes lead to longer deliberation, but less calibrated reliance, with participants increasingly deferring to incorrect AI advice as decision time increased. These findings highlight that increased effort under higher stakes does not necessarily improve reliance calibration and show the importance of accounting for stakes when evaluating human-AI decision-making.