Modular Reinforcement Learning For Cooperative Swarms
This work provides a memory-efficient solution for reinforcement learning in robot swarms, which is important for computationally-limited robots.
The paper addresses the challenge of representing combinatorial interaction states in cooperative robot swarms for reinforcement learning. It proposes a modular representation where each state feature is handled by a separate learning procedure, and demonstrates improved memory efficiency and performance in simulated foraging tasks.
A cooperative robot swarm is a collective of computationally-limited robots that share a common goal. Each robot can only interact with a small subset of its peers, without knowing how this affects the collective utility. Recent advances in distributed multi-agent reinforcement learning have demonstrated that it is possible for robots to learn how to interact effectively with others, in a manner that is aligned with the common goal, despite each robot learning independently of others. However, this requires each robot to represent a potentially combinatorial number of interaction states, challenging the memory capabilities of the robots. This paper proposes an alternative approach for representing spatial interaction states for multi-robot reinforcement learning in swarms. A modular (decomposed) representation is used, where each feature of the state is handled by a separate learning procedure, and the results aggregated. We demonstrate the efficacy of the approach in numerous experiments with simulated robot swarms carrying out foraging.