From Accuracy to Readiness: Metrics and Benchmarks for Human-AI Decision-Making
This work addresses the need for better evaluation metrics in human-AI decision-making to improve safety and accountability, representing an incremental advancement by systematizing existing concepts into a framework.
The paper tackles the problem that current evaluation practices for AI systems in human decision-making focus on model accuracy rather than team readiness, leading to failures from miscalibrated reliance. It proposes a measurement framework with a four-part taxonomy of metrics to assess human-AI collaboration through interaction traces, aiming to enable more comparable benchmarks and safer collaboration.
Artificial intelligence (AI) systems are deployed as collaborators in human decision-making. Yet, evaluation practices focus primarily on model accuracy rather than whether human-AI teams are prepared to collaborate safely and effectively. Empirical evidence shows that many failures arise from miscalibrated reliance, including overuse when AI is wrong and underuse when it is helpful. This paper proposes a measurement framework for evaluating human-AI decision-making centered on team readiness. We introduce a four part taxonomy of evaluation metrics spanning outcomes, reliance behavior, safety signals, and learning over time, and connect these metrics to the Understand-Control-Improve (U-C-I) lifecycle of human-AI onboarding and collaboration. By operationalizing evaluation through interaction traces rather than model properties or self-reported trust, our framework enables deployment-relevant assessment of calibration, error recovery, and governance. We aim to support more comparable benchmarks and cumulative research on human-AI readiness, advancing safer and more accountable human-AI collaboration.