Joint Attention for Multi-Agent Coordination and Social Learning
This addresses the challenge of efficient coordination and learning in multi-agent systems, with potential applications in robotics and AI, though it is incremental as it builds on existing deep reinforcement learning methods.
The paper tackled the problem of improving multi-agent coordination and social learning by introducing a joint attention mechanism, showing that it reduces the exponential cost of exploring joint action spaces and leads to higher performance than a centralized critic baseline across multiple environments.
Joint attention - the ability to purposefully coordinate attention with another agent, and mutually attend to the same thing -- is a critical component of human social cognition. In this paper, we ask whether joint attention can be useful as a mechanism for improving multi-agent coordination and social learning. We first develop deep reinforcement learning (RL) agents with a recurrent visual attention architecture. We then train agents to minimize the difference between the attention weights that they apply to the environment at each timestep, and the attention of other agents. Our results show that this joint attention incentive improves agents' ability to solve difficult coordination tasks, by reducing the exponential cost of exploring the joint multi-agent action space. Joint attention leads to higher performance than a competitive centralized critic baseline across multiple environments. Further, we show that joint attention enhances agents' ability to learn from experts present in their environment, even when completing hard exploration tasks that do not require coordination. Taken together, these findings suggest that joint attention may be a useful inductive bias for multi-agent learning.