Artificial Generals Intelligence: Mastering Generals.io with Reinforcement Learning
This provides an accessible yet challenging benchmark for multi-agent reinforcement learning research, though it is incremental as it builds on existing methods like supervised pre-training and self-play.
The authors tackled the problem of advancing multi-agent reinforcement learning by creating a real-time strategy game environment based on Generals.io and training a reference agent that reached the top 0.003% of the human leaderboard in 36 hours on a single H100 GPU.
We introduce a real-time strategy game environment based on Generals.io, a game with thousands of weekly active players. Our environment is fully compatible with Gymnasium and PettingZoo and is capable of running thousands of frames per second on commodity hardware. We also present a reference agent, trained with supervised pre-training and self-play, which reached the top 0.003% of the 1v1 human leaderboard after only 36 hours on a single H100 GPU. To accelerate learning, we incorporate potential-based reward shaping and memory features. Our contributions of a modular RTS benchmark and a competitive baseline agent provide an accessible yet challenging platform for advancing multi-agent reinforcement learning research. The documented code, together with examples and tutorials, is available at https://github.com/strakam/generals-bots.