A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry
This work addresses the problem of better understanding human decision-making and neuropsychiatric conditions for researchers in AI and psychology, but it is incremental as it builds on existing reinforcement learning methods.
The authors tackled the problem of modeling human decision-making by proposing a two-stream reinforcement learning framework that processes positive and negative rewards separately, incorporating reward-processing biases. The result showed that their Split-QL model outperformed standard methods like Q-Learning and SARSA, as well as Double Q-Learning, on simulated tasks, a real-world gambling dataset, and the Pac-Man game.
Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-processing biases -- an important component of human decision making which can help us better understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems, as well as various neuropsychiatric conditions associated with disruptions in normal reward processing. From the computational perspective, we observe that the proposed Split-QL model and its clinically inspired variants consistently outperform standard Q-Learning and SARSA methods, as well as recently proposed Double Q-Learning approaches, on simulated tasks with particular reward distributions, a real-world dataset capturing human decision-making in gambling tasks, and the Pac-Man game in a lifelong learning setting across different reward stationarities.