Reducing Redundant Computation in Multi-Agent Coordination through Locally Centralized Execution
This addresses efficiency issues in multi-agent coordination for applications like robotics or gaming, though it is incremental as it builds on existing decentralized execution methods.
The paper tackles the problem of redundant computation in multi-agent reinforcement learning by introducing a locally centralized execution framework with a team-transformer architecture and leadership shift mechanism, resulting in reduced computation without decreasing rewards and faster learning convergence.
In multi-agent reinforcement learning, decentralized execution is a common approach, yet it suffers from the redundant computation problem. This occurs when multiple agents redundantly perform the same or similar computation due to overlapping observations. To address this issue, this study introduces a novel method referred to as locally centralized team transformer (LCTT). LCTT establishes a locally centralized execution framework where selected agents serve as leaders, issuing instructions, while the rest agents, designated as workers, act as these instructions without activating their policy networks. For LCTT, we proposed the team-transformer (T-Trans) architecture that allows leaders to provide specific instructions to each worker, and the leadership shift mechanism that allows agents autonomously decide their roles as leaders or workers. Our experimental results demonstrate that the proposed method effectively reduces redundant computation, does not decrease reward levels, and leads to faster learning convergence.