MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning
This addresses the challenge of building generalist agents for multi-agent scenarios, enabling zero-shot performance across diverse tasks, though it is incremental in extending single-agent methods to multi-agent settings.
The paper tackled the problem of zero-shot transfer in multi-agent decision making by proposing MaskMA, a mask-based collaborative learning framework, which achieved a 77.8% average zero-shot win rate on 60 unseen test maps in SMAC.
Building a single generalist agent with strong zero-shot capability has recently sparked significant advancements. However, extending this capability to multi-agent decision making scenarios presents challenges. Most current works struggle with zero-shot transfer, due to two challenges particular to the multi-agent settings: (a) a mismatch between centralized training and decentralized execution; and (b) difficulties in creating generalizable representations across diverse tasks due to varying agent numbers and action spaces. To overcome these challenges, we propose a Mask-Based collaborative learning framework for Multi-Agent decision making (MaskMA). Firstly, we propose to randomly mask part of the units and collaboratively learn the policies of unmasked units to handle the mismatch. In addition, MaskMA integrates a generalizable action representation by dividing the action space into intrinsic actions solely related to the unit itself and interactive actions involving interactions with other units. This flexibility allows MaskMA to tackle tasks with varying agent numbers and thus different action spaces. Extensive experiments in SMAC reveal MaskMA, with a single model trained on 11 training maps, can achieve an impressive 77.8% average zero-shot win rate on 60 unseen test maps by decentralized execution, while also performing effectively on other types of downstream tasks (e.g., varied policies collaboration, ally malfunction, and ad hoc team play).