Inverse Reinforcement Learning in Swarm Systems
This work addresses the challenge of learning behavioral models from demonstrations in large-scale homogeneous multi-agent systems, which is an incremental advancement in the field.
The authors tackled the problem of applying inverse reinforcement learning to multi-agent swarm systems by introducing the swarMDP framework and reducing it to a single-agent problem, demonstrating that it can produce meaningful local reward models to replicate global dynamics in two example systems.
Inverse reinforcement learning (IRL) has become a useful tool for learning behavioral models from demonstration data. However, IRL remains mostly unexplored for multi-agent systems. In this paper, we show how the principle of IRL can be extended to homogeneous large-scale problems, inspired by the collective swarming behavior of natural systems. In particular, we make the following contributions to the field: 1) We introduce the swarMDP framework, a sub-class of decentralized partially observable Markov decision processes endowed with a swarm characterization. 2) Exploiting the inherent homogeneity of this framework, we reduce the resulting multi-agent IRL problem to a single-agent one by proving that the agent-specific value functions in this model coincide. 3) To solve the corresponding control problem, we propose a novel heterogeneous learning scheme that is particularly tailored to the swarm setting. Results on two example systems demonstrate that our framework is able to produce meaningful local reward models from which we can replicate the observed global system dynamics.