Multi-Agent Coordination via Multi-Level Communication
This addresses coordination challenges in multi-agent systems, such as robotics or gaming, but appears incremental as it builds on existing communication-based methods.
The paper tackles the coordination problem in multi-agent systems with partial observability and stochasticity by proposing a multi-level communication scheme called Sequential Communication (SeqComm), which uses asynchronous decision-making and two communication phases to improve performance, and empirically shows it outperforms existing methods in cooperative tasks.
The partial observability and stochasticity in multi-agent settings can be mitigated by accessing more information about others via communication. However, the coordination problem still exists since agents cannot communicate actual actions with each other at the same time due to the circular dependencies. In this paper, we propose a novel multi-level communication scheme, Sequential Communication (SeqComm). SeqComm treats agents asynchronously (the upper-level agents make decisions before the lower-level ones) and has two communication phases. In the negotiation phase, agents determine the priority of decision-making by communicating hidden states of observations and comparing the value of intention, obtained by modeling the environment dynamics. In the launching phase, the upper-level agents take the lead in making decisions and then communicate their actions with the lower-level agents. Theoretically, we prove the policies learned by SeqComm are guaranteed to improve monotonically and converge. Empirically, we show that SeqComm outperforms existing methods in various cooperative multi-agent tasks.