Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale
This provides a new benchmark for MARL researchers to evaluate long-term dependencies and generalization, though it is incremental as an extension of an existing environment.
The authors tackled the lack of challenging benchmarks for multi-agent reinforcement learning (MARL) by introducing Craftax-MA and Craftax-Coop, extensions of the Craftax environment that support multiple agents and complex cooperation, with training runs of 250 million interactions completing in under an hour. They demonstrated that existing algorithms struggle with long-horizon credit assignment, exploration, and cooperation in these benchmarks.
Progress in multi-agent reinforcement learning (MARL) requires challenging benchmarks that assess the limits of current methods. However, existing benchmarks often target narrow short-horizon challenges that do not adequately stress the long-term dependencies and generalization capabilities inherent in many multi-agent systems. To address this, we first present \textit{Craftax-MA}: an extension of the popular open-ended RL environment, Craftax, that supports multiple agents and evaluates a wide range of general abilities within a single environment. Written in JAX, \textit{Craftax-MA} is exceptionally fast with a training run using 250 million environment interactions completing in under an hour. To provide a more compelling challenge for MARL, we also present \textit{Craftax-Coop}, an extension introducing heterogeneous agents, trading and more mechanics that require complex cooperation among agents for success. We provide analysis demonstrating that existing algorithms struggle with key challenges in this benchmark, including long-horizon credit assignment, exploration and cooperation, and argue for its potential to drive long-term research in MARL.