Conformal Prediction Sets Improve Human Decision Making
This work addresses the challenge of enhancing human-AI collaboration in decision-making tasks, though it is incremental as it builds on existing conformal prediction methods.
The study tackled the problem of improving human decision-making with AI by testing conformal prediction sets, which provide calibrated uncertainty estimates and alternative answers, and found that they significantly improved human accuracy compared to fixed-size sets with the same coverage.
In response to everyday queries, humans explicitly signal uncertainty and offer alternative answers when they are unsure. Machine learning models that output calibrated prediction sets through conformal prediction mimic this human behaviour; larger sets signal greater uncertainty while providing alternatives. In this work, we study the usefulness of conformal prediction sets as an aid for human decision making by conducting a pre-registered randomized controlled trial with conformal prediction sets provided to human subjects. With statistical significance, we find that when humans are given conformal prediction sets their accuracy on tasks improves compared to fixed-size prediction sets with the same coverage guarantee. The results show that quantifying model uncertainty with conformal prediction is helpful for human-in-the-loop decision making and human-AI teams.