ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination
This addresses the challenge of inadequate evaluation for ZSC algorithms, which is crucial for researchers in cooperative AI, though it is incremental as it builds on existing ZSC frameworks.
The paper tackles the problem of evaluating zero-shot coordination (ZSC) algorithms in multi-agent reinforcement learning by introducing ZSC-Eval, a toolkit and benchmark that generates diverse evaluation partners and uses metrics like BR-Prox to measure generalization, leading to novel empirical findings in environments like Overcooked and Google Research Football.
Zero-shot coordination (ZSC) is a new cooperative multi-agent reinforcement learning (MARL) challenge that aims to train an ego agent to work with diverse, unseen partners during deployment. The significant difference between the deployment-time partners' distribution and the training partners' distribution determined by the training algorithm makes ZSC a unique out-of-distribution (OOD) generalization challenge. The potential distribution gap between evaluation and deployment-time partners leads to inadequate evaluation, which is exacerbated by the lack of appropriate evaluation metrics. In this paper, we present ZSC-Eval, the first evaluation toolkit and benchmark for ZSC algorithms. ZSC-Eval consists of: 1) Generation of evaluation partner candidates through behavior-preferring rewards to approximate deployment-time partners' distribution; 2) Selection of evaluation partners by Best-Response Diversity (BR-Div); 3) Measurement of generalization performance with various evaluation partners via the Best-Response Proximity (BR-Prox) metric. We use ZSC-Eval to benchmark ZSC algorithms in Overcooked and Google Research Football environments and get novel empirical findings. We also conduct a human experiment of current ZSC algorithms to verify the ZSC-Eval's consistency with human evaluation. ZSC-Eval is now available at https://github.com/sjtu-marl/ZSC-Eval.