Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module
This addresses a practical problem for real-world multi-agent systems where agent numbers are dynamic, though it appears incremental as it builds on existing meta-RL and communication methods.
The paper tackles the challenge of designing communication mechanisms for multi-agent reinforcement learning when the number of agents varies or grows, proposing a meta-RL framework with a Communication Pattern Recognition module that generalizes to unseen larger numbers of agents and allows agent counts to change between episodes.
Designing an effective communication mechanism among agents in reinforcement learning has been a challenging task, especially for real-world applications. The number of agents can grow or an environment sometimes needs to interact with a changing number of agents in real-world scenarios. To this end, a multi-agent framework needs to handle various scenarios of agents, in terms of both scales and dynamics, for being practical to real-world applications. We formulate the multi-agent environment with a different number of agents as a multi-tasking problem and propose a meta reinforcement learning (meta-RL) framework to tackle this problem. The proposed framework employs a meta-learned Communication Pattern Recognition (CPR) module to identify communication behavior and extract information that facilitates the training process. Experimental results are poised to demonstrate that the proposed framework (a) generalizes to an unseen larger number of agents and (b) allows the number of agents to change between episodes. The ablation study is also provided to reason the proposed CPR design and show such design is effective.