Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
This work addresses the challenge of enhancing human-AI collaboration in high-stakes applications by mitigating cognitive biases, though it is incremental as it builds on existing research in cognitive science and AI-assisted decision-making.
The paper tackles the problem of cognitive biases, such as anchoring bias, negatively affecting human-AI collaborative decision-making by mathematically modeling these biases and proposing a time-based de-anchoring strategy, which is validated through user experiments to improve collaborative performance, particularly when the AI model has low confidence and is incorrect.
Several strands of research have aimed to bridge the gap between artificial intelligence (AI) and human decision-makers in AI-assisted decision-making, where humans are the consumers of AI model predictions and the ultimate decision-makers in high-stakes applications. However, people's perception and understanding are often distorted by their cognitive biases, such as confirmation bias, anchoring bias, availability bias, to name a few. In this work, we use knowledge from the field of cognitive science to account for cognitive biases in the human-AI collaborative decision-making setting, and mitigate their negative effects on collaborative performance. To this end, we mathematically model cognitive biases and provide a general framework through which researchers and practitioners can understand the interplay between cognitive biases and human-AI accuracy. We then focus specifically on anchoring bias, a bias commonly encountered in human-AI collaboration. We implement a time-based de-anchoring strategy and conduct our first user experiment that validates its effectiveness in human-AI collaborative decision-making. With this result, we design a time allocation strategy for a resource-constrained setting that achieves optimal human-AI collaboration under some assumptions. We, then, conduct a second user experiment which shows that our time allocation strategy with explanation can effectively de-anchor the human and improve collaborative performance when the AI model has low confidence and is incorrect.