Community-based Multi-Agent Reinforcement Learning with Transfer and Active Exploration
This work addresses coordination challenges in multi-agent systems for applications like robotics or social networks, offering a novel integration of community structure with transfer and active learning.
The paper tackles the problem of multi-agent reinforcement learning in networks with latent community structures by proposing a community-based framework that captures flexible coordination patterns, achieving convergence guarantees under linear function approximation.
We propose a new framework for multi-agent reinforcement learning (MARL), where the agents cooperate in a time-evolving network with latent community structures and mixed memberships. Unlike traditional neighbor-based or fixed interaction graphs, our community-based framework captures flexible and abstract coordination patterns by allowing each agent to belong to multiple overlapping communities. Each community maintains shared policy and value functions, which are aggregated by individual agents according to personalized membership weights. We also design actor-critic algorithms that exploit this structure: agents inherit community-level estimates for policy updates and value learning, enabling structured information sharing without requiring access to other agents' policies. Importantly, our approach supports both transfer learning by adapting to new agents or tasks via membership estimation, and active learning by prioritizing uncertain communities during exploration. Theoretically, we establish convergence guarantees under linear function approximation for both actor and critic updates. To our knowledge, this is the first MARL framework that integrates community structure, transferability, and active learning with provable guarantees.