Learning Multi-agent Communication under Limited-bandwidth Restriction for Internet Packet Routing
This addresses a practical limitation for deploying multi-agent systems in real-world applications like internet packet routing, though it is incremental as it builds on existing DRL-based communication methods.
The paper tackled the problem of multi-agent communication under limited-bandwidth restrictions, which existing DRL methods ignore by sending too many messages. It proposed a gating mechanism that prunes over 80% of messages with minimal performance loss and outperforms state-of-the-art methods in packet routing and benchmark tasks.
Communication is an important factor for the big multi-agent world to stay organized and productive. Recently, the AI community has applied the Deep Reinforcement Learning (DRL) to learn the communication strategy and the control policy for multiple agents. However, when implementing the communication for real-world multi-agent applications, there is a more practical limited-bandwidth restriction, which has been largely ignored by the existing DRL-based methods. Specifically, agents trained by most previous methods keep sending messages incessantly in every control cycle; due to emitting too many messages, these methods are unsuitable to be applied to the real-world systems that have a limited bandwidth to transmit the messages. To handle this problem, we propose a gating mechanism to adaptively prune unprofitable messages. Results show that the gating mechanism can prune more than 80% messages with little damage to the performance. Moreover, our method outperforms several state-of-the-art DRL-based and rule-based methods by a large margin in both the real-world packet routing tasks and four benchmark tasks.