The Value of Information in Human-AI Decision-making
This work addresses the challenge of enhancing collaborative performance in human-AI pairings for decision-making tasks, such as medical diagnosis and deepfake detection, though it appears incremental in building on existing explanation methods.
The paper tackles the problem of improving human-AI decision-making by developing a framework to characterize the value of complementary information and a novel explanation technique (ILIV-SHAP). The result shows that presenting ILIV-SHAP with AI predictions leads to greater error reductions compared to non-AI assisted decisions and vanilla SHAP.
Multiple agents are increasingly combined to make decisions with the expectation of achieving complementary performance, where the decisions they make together outperform those made individually. However, knowing how to improve the performance of collaborating agents requires knowing what information and strategies each agent employs. With a focus on human-AI pairings, we contribute a decision-theoretic framework for characterizing the value of information. By defining complementary information, our approach identifies opportunities for agents to better exploit available information in AI-assisted decision workflows. We present a novel explanation technique (ILIV-SHAP) that adapts SHAP explanations to highlight human-complementing information. We validate the effectiveness of ACIV and ILIV-SHAP through a study of human-AI decision-making, and demonstrate the framework on examples from chest X-ray diagnosis and deepfake detection. We find that presenting ILIV-SHAP with AI predictions leads to reliably greater reductions in error over non-AI assisted decisions more than vanilla SHAP.