Integrating Expert Judgment and Algorithmic Decision Making: An Indistinguishability Framework
This addresses the challenge of integrating human expertise with AI in prediction tasks, offering a principled method for selective feedback incorporation, though it is incremental in refining collaboration frameworks.
The paper tackles the problem of human-AI collaboration by introducing a framework that uses human judgment to handle inputs that are algorithmically indistinguishable, showing it provably improves algorithmic predictors and quantifying this improvement. In an emergency room triage case study, it finds physician judgments provide signal unreplicable by algorithms, leading to decision rules that leverage complementary strengths.
We introduce a novel framework for human-AI collaboration in prediction and decision tasks. Our approach leverages human judgment to distinguish inputs which are algorithmically indistinguishable, or "look the same" to any feasible predictive algorithm. We argue that this framing clarifies the problem of human-AI collaboration in prediction and decision tasks, as experts often form judgments by drawing on information which is not encoded in an algorithm's training data. Algorithmic indistinguishability yields a natural test for assessing whether experts incorporate this kind of "side information", and further provides a simple but principled method for selectively incorporating human feedback into algorithmic predictions. We show that this method provably improves the performance of any feasible algorithmic predictor and precisely quantify this improvement. We demonstrate the utility of our framework in a case study of emergency room triage decisions, where we find that although algorithmic risk scores are highly competitive with physicians, there is strong evidence that physician judgments provide signal which could not be replicated by any predictive algorithm. This insight yields a range of natural decision rules which leverage the complementary strengths of human experts and predictive algorithms.