Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution
This addresses the challenge of efficient coordination in heterogeneous multi-agent systems, which is incremental as it builds on prior work by extending zero-shot coordination to heterogeneous settings.
The paper tackles the problem of zero-shot coordination with unseen partners in heterogeneous multi-agent tasks, proposing a coevolution-based method that coevolves populations of agents and partners, and demonstrates its effectiveness on various heterogeneous tasks.
Generating agents that can achieve zero-shot coordination (ZSC) with unseen partners is a new challenge in cooperative multi-agent reinforcement learning (MARL). Recently, some studies have made progress in ZSC by exposing the agents to diverse partners during the training process. They usually involve self-play when training the partners, implicitly assuming that the tasks are homogeneous. However, many real-world tasks are heterogeneous, and hence previous methods may be inefficient. In this paper, we study the heterogeneous ZSC problem for the first time and propose a general method based on coevolution, which coevolves two populations of agents and partners through three sub-processes: pairing, updating and selection. Experimental results on various heterogeneous tasks highlight the necessity of considering the heterogeneous setting and demonstrate that our proposed method is a promising solution for heterogeneous ZSC tasks.