Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making
This work addresses the problem of improving human-AI collaboration in decision-making, particularly in healthcare domains like stroke rehabilitation, though it appears incremental by building on existing uncertainty methods.
The paper tackled the challenge of fostering appropriate human reliance on AI in decision-making by investigating distance-based uncertainty scores for task delegation, finding that these scores outperformed traditional probability-based scores in identifying uncertain cases and led to an 8.20% higher rate of correct decisions in a study with health professionals and students.
Despite the growing promise of artificial intelligence (AI) in supporting decision-making across domains, fostering appropriate human reliance on AI remains a critical challenge. In this paper, we investigate the utility of exploring distance-based uncertainty scores for task delegation to AI and describe how these scores can be visualized through embedding representations for human-AI decision-making. After developing an AI-based system for physical stroke rehabilitation assessment, we conducted a study with 19 health professionals and 10 students in medicine/health to understand the effect of exploring distance-based uncertainty scores on users' reliance on AI. Our findings showed that distance-based uncertainty scores outperformed traditional probability-based uncertainty scores in identifying uncertain cases. In addition, after exploring confidence scores for task delegation and reviewing embedding-based visualizations of distance-based uncertainty scores, participants achieved an 8.20% higher rate of correct decisions, a 7.15% higher rate of changing their decisions to correct ones, and a 7.14% lower rate of incorrect changes after reviewing AI outputs than those reviewing probability-based uncertainty scores ($p<0.01$). Our findings highlight the potential of distance-based uncertainty scores to enhance decision accuracy and appropriate reliance on AI while discussing ongoing challenges for human-AI collaborative decision-making.