Mean Field Games Flock! The Reinforcement Learning Way
This addresses the challenge of scalable flocking simulations for robotics or multi-agent systems, though it appears incremental as it builds on existing MFG and RL techniques.
The authors tackled the problem of enabling a large number of agents to learn flocking behavior by formulating it as a Mean Field Game and combining Deep Reinforcement Learning with Normalizing Flows, resulting in a tractable solution that learns multi-group or high-dimensional flocking with obstacles.
We present a method enabling a large number of agents to learn how to flock, which is a natural behavior observed in large populations of animals. This problem has drawn a lot of interest but requires many structural assumptions and is tractable only in small dimensions. We phrase this problem as a Mean Field Game (MFG), where each individual chooses its acceleration depending on the population behavior. Combining Deep Reinforcement Learning (RL) and Normalizing Flows (NF), we obtain a tractable solution requiring only very weak assumptions. Our algorithm finds a Nash Equilibrium and the agents adapt their velocity to match the neighboring flock's average one. We use Fictitious Play and alternate: (1) computing an approximate best response with Deep RL, and (2) estimating the next population distribution with NF. We show numerically that our algorithm learn multi-group or high-dimensional flocking with obstacles.