Brain-Like Replay Naturally Emerges in Reinforcement Learning Agents
This research addresses the conditions and advantages of replay for developing efficient reinforcement learning, but it appears incremental as it builds on existing modular ANN approaches.
The study tackled the problem of generating replay in reinforcement learning agents by developing a modular model that naturally produces replay, proving it helps complete tasks and reproduces biological phenomena.
Replay is a powerful strategy to promote learning in artificial intelligence and the brain. However, the conditions to generate it and its functional advantages have not been fully recognized. In this study, we develop a modular reinforcement learning model that could generate replay. We prove that replay generated in this way helps complete the task. We also analyze the information contained in the representation and provide a mechanism for how replay makes a difference. Our design avoids complex assumptions and enables replay to emerge naturally within a task-optimized paradigm. Our model also reproduces key phenomena observed in biological agents. This research explores the structural biases in modular ANN to generate replay and its potential utility in developing efficient RL.