Learning Multi-Agent Coordination through Connectivity-driven Communication
This work addresses the challenge of effective multi-agent coordination in AI, offering a novel method for learning communication strategies, though it is incremental in building upon existing graph-based and attention mechanisms.
The paper tackles the problem of learning collaborative policies in multi-agent systems by introducing Connectivity Driven Communication (CDC), a deep reinforcement learning approach that enables agents to dynamically learn communication and coordination through experience, achieving superior performance on cooperative navigation tasks compared to existing algorithms.
In artificial multi-agent systems, the ability to learn collaborative policies is predicated upon the agents' communication skills: they must be able to encode the information received from the environment and learn how to share it with other agents as required by the task at hand. We present a deep reinforcement learning approach, Connectivity Driven Communication (CDC), that facilitates the emergence of multi-agent collaborative behaviour only through experience. The agents are modelled as nodes of a weighted graph whose state-dependent edges encode pair-wise messages that can be exchanged. We introduce a graph-dependent attention mechanisms that controls how the agents' incoming messages are weighted. This mechanism takes into full account the current state of the system as represented by the graph, and builds upon a diffusion process that captures how the information flows on the graph. The graph topology is not assumed to be known a priori, but depends dynamically on the agents' observations, and is learnt concurrently with the attention mechanism and policy in an end-to-end fashion. Our empirical results show that CDC is able to learn effective collaborative policies and can over-perform competing learning algorithms on cooperative navigation tasks.