Learning Complementary Policies for Human-AI Teams
This addresses the problem of inefficient human-AI collaboration in complex management settings, representing a novel method for a known bottleneck rather than incremental work.
This paper tackles the challenge of improving human-AI team performance in decision-making tasks by developing a deferral collaboration approach that strategically allocates instances between humans and AI. The method significantly outperforms independent human and algorithmic decision-making, achieving substantial performance improvements by routing only a small fraction of instances to humans.
This paper tackles the critical challenge of human-AI complementarity in decision-making. Departing from the traditional focus on algorithmic performance in favor of performance of the human-AI team, and moving past the framing of collaboration as classification to focus on decision-making tasks, we introduce a novel approach to policy learning. Specifically, we develop a robust solution for human-AI collaboration when outcomes are only observed under assigned actions. We propose a deferral collaboration approach that maximizes decision rewards by exploiting the distinct strengths of humans and AI, strategically allocating instances among them. Critically, our method is robust to misspecifications in both the human behavior and reward models. Leveraging the insight that performance gains stem from divergent human and AI behavioral patterns, we demonstrate, using synthetic and real human responses, that our proposed method significantly outperforms independent human and algorithmic decision-making. Moreover, we show that substantial performance improvements are achievable by routing only a small fraction of instances to human decision-makers, highlighting the potential for efficient and effective human-AI collaboration in complex management settings.