Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learning
This addresses the problem of understanding collective dynamics in complex systems like fish schools for researchers, though it appears incremental as it builds on existing inverse reinforcement learning techniques.
The study tackled the challenge of discovering individual objectives in collective behavior by introducing an off-policy inverse multi-agent reinforcement learning algorithm (IMARL), which automatically uncovers reward functions and learns effective policies, achieving promising results in domains like OpenAI gym and multi-agent schooling models.
The discovery of individual objectives in collective behavior of complex dynamical systems such as fish schools and bacteria colonies is a long-standing challenge. Inverse reinforcement learning is a potent approach for addressing this challenge but its applicability to dynamical systems, involving continuous state-action spaces and multiple interacting agents, has been limited. In this study, we tackle this challenge by introducing an off-policy inverse multi-agent reinforcement learning algorithm (IMARL). Our approach combines the ReF-ER techniques with guided cost learning. By leveraging demonstrations, our algorithm automatically uncovers the reward function and learns an effective policy for the agents. Through extensive experimentation, we demonstrate that the proposed policy captures the behavior observed in the provided data, and achieves promising results across problem domains including single agent models in the OpenAI gym and multi-agent models of schooling behavior. The present study shows that the proposed IMARL algorithm is a significant step towards understanding collective dynamics from the perspective of its constituents, and showcases its value as a tool for studying complex physical systems exhibiting collective behaviour.