Teaching Humans When To Defer to a Classifier via Exemplars
This work addresses the challenge of human-AI collaboration by ensuring humans learn effective mental models, which is incremental as it builds on existing teaching methods with a novel parameterization.
The paper tackles the problem of teaching human decision makers when to rely on an AI classifier by proposing an exemplar-based strategy to help them learn valid mental models of the agent's strengths and weaknesses, resulting in improved task performance when workers apply the correct lessons, as validated on a multi-hop question answering task with crowd workers.
Expert decision makers are starting to rely on data-driven automated agents to assist them with various tasks. For this collaboration to perform properly, the human decision maker must have a mental model of when and when not to rely on the agent. In this work, we aim to ensure that human decision makers learn a valid mental model of the agent's strengths and weaknesses. To accomplish this goal, we propose an exemplar-based teaching strategy where humans solve the task with the help of the agent and try to formulate a set of guidelines of when and when not to defer. We present a novel parameterization of the human's mental model of the AI that applies a nearest neighbor rule in local regions surrounding the teaching examples. Using this model, we derive a near-optimal strategy for selecting a representative teaching set. We validate the benefits of our teaching strategy on a multi-hop question answering task using crowd workers and find that when workers draw the right lessons from the teaching stage, their task performance improves, we furthermore validate our method on a set of synthetic experiments.