Predicting and Understanding Human Action Decisions: Insights from Large Language Models and Cognitive Instance-Based Learning
This work addresses the challenge of modeling human decision-making for AI-assisted systems, though it is incremental in integrating existing methods.
The paper tackled the problem of predicting human behavior in sequential decision-making tasks by comparing large language models (LLMs) with a cognitive instance-based learning model, finding that LLMs excel at incorporating feedback for accuracy while the cognitive model better captures human exploratory behaviors and loss aversion bias.
Large Language Models (LLMs) have demonstrated their capabilities across various tasks, from language translation to complex reasoning. Understanding and predicting human behavior and biases are crucial for artificial intelligence (AI) assisted systems to provide useful assistance, yet it remains an open question whether these models can achieve this. This paper addresses this gap by leveraging the reasoning and generative capabilities of the LLMs to predict human behavior in two sequential decision-making tasks. These tasks involve balancing between exploitative and exploratory actions and handling delayed feedback, both essential for simulating real-life decision processes. We compare the performance of LLMs with a cognitive instance-based learning (IBL) model, which imitates human experiential decision-making. Our findings indicate that LLMs excel at rapidly incorporating feedback to enhance prediction accuracy. In contrast, the cognitive IBL model better accounts for human exploratory behaviors and effectively captures loss aversion bias, i.e., the tendency to choose a sub-optimal goal with fewer step-cost penalties rather than exploring to find the optimal choice, even with limited experience. The results highlight the benefits of integrating LLMs with cognitive architectures, suggesting that this synergy could enhance the modeling and understanding of complex human decision-making patterns.