Selectively Providing Reliance Calibration Cues With Reliance Prediction
This addresses reliance calibration for users of AI systems, but it is incremental as it builds on prior work by focusing on selective cue provision.
The paper tackles the problem of over/under-reliance in human-AI collaboration by proposing Pred-RC, a method that selectively provides reliance calibration cues based on predictions of human reliance, resulting in successful calibration with a reduced number of cues.
For effective collaboration between humans and intelligent agents that employ machine learning for decision-making, humans must understand what agents can and cannot do to avoid over/under-reliance. A solution to this problem is adjusting human reliance through communication using reliance calibration cues (RCCs) to help humans assess agents' capabilities. Previous studies typically attempted to calibrate reliance by continuously presenting RCCs, and when an agent should provide RCCs remains an open question. To answer this, we propose Pred-RC, a method for selectively providing RCCs. Pred-RC uses a cognitive reliance model to predict whether a human will assign a task to an agent. By comparing the prediction results for both cases with and without an RCC, Pred-RC evaluates the influence of the RCC on human reliance. We tested Pred-RC in a human-AI collaboration task and found that it can successfully calibrate human reliance with a reduced number of RCCs.