High Performance Simulation for Scalable Multi-Agent Reinforcement Learning
This enables scalable and efficient training for complex simulations, though it is incremental as it builds on existing multi-agent reinforcement learning methods with a focus on performance improvements.
The paper tackles the problem of limited scale in multi-agent reinforcement learning environments by introducing Vogue, a high-performance agent-based model framework that supports thousands to tens of thousands of interacting agents on GPUs, achieving training times in minutes to hours.
Multi-agent reinforcement learning experiments and open-source training environments are typically limited in scale, supporting tens or sometimes up to hundreds of interacting agents. In this paper we demonstrate the use of Vogue, a high performance agent based model (ABM) framework. Vogue serves as a multi-agent training environment, supporting thousands to tens of thousands of interacting agents while maintaining high training throughput by running both the environment and reinforcement learning (RL) agents on the GPU. High performance multi-agent environments at this scale have the potential to enable the learning of robust and flexible policies for use in ABMs and simulations of complex systems. We demonstrate training performance with two newly developed, large scale multi-agent training environments. Moreover, we show that these environments can train shared RL policies on time-scales of minutes and hours.