Enhancing Multi-Agent Collaboration with Attention-Based Actor-Critic Policies
This work addresses multi-agent collaboration for reinforcement learning applications, representing an incremental improvement with novel attention mechanisms.
This paper tackled the problem of enhancing multi-agent collaboration in cooperative environments by introducing the Team-Attention-Actor-Critic (TAAC) algorithm, which demonstrated superior performance in a simulated soccer environment with improved win rates, goal differentials, and other metrics compared to benchmarks.
This paper introduces Team-Attention-Actor-Critic (TAAC), a reinforcement learning algorithm designed to enhance multi-agent collaboration in cooperative environments. TAAC employs a Centralized Training/Centralized Execution scheme incorporating multi-headed attention mechanisms in both the actor and critic. This design facilitates dynamic, inter-agent communication, allowing agents to explicitly query teammates, thereby efficiently managing the exponential growth of joint-action spaces while ensuring a high degree of collaboration. We further introduce a penalized loss function which promotes diverse yet complementary roles among agents. We evaluate TAAC in a simulated soccer environment against benchmark algorithms representing other multi-agent paradigms, including Proximal Policy Optimization and Multi-Agent Actor-Attention-Critic. We find that TAAC exhibits superior performance and enhanced collaborative behaviors across a variety of metrics (win rates, goal differentials, Elo ratings, inter-agent connectivity, balanced spatial distributions, and frequent tactical interactions such as ball possession swaps).