Learning Structured Communication for Multi-agent Reinforcement Learning
This work addresses the challenge of scalable and efficient communication for multi-agent systems, which is incremental as it builds on existing MARL methods by introducing a learnable hierarchical structure.
The paper tackles the problem of designing efficient communication topologies in multi-agent reinforcement learning by proposing the Learning Structured Communication (LSC) framework, which adaptively groups agents into hierarchical formations and uses a hierarchical graph neural network for message propagation, resulting in high communication efficiency, scalability, and global cooperation capability in experiments.
This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We summarize the general categories of topology for communication structures in MARL literature, which are often manually specified. Then we propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology. Our framework allows for adaptive agent grouping to form different hierarchical formations over episodes, which is generated by an auxiliary task combined with a hierarchical routing protocol. Given each formed topology, a hierarchical graph neural network is learned to enable effective message information generation and propagation among inter- and intra-group communications. In contrast to existing communication mechanisms, our method has an explicit while learnable design for hierarchical communication. Experiments on challenging tasks show the proposed LSC enjoys high communication efficiency, scalability, and global cooperation capability.