Learning to Interact in World Latent for Team Coordination
This work addresses coordination challenges in multi-agent systems, offering a novel representation that enhances existing algorithms, though it is incremental in building on prior MARL methods.
The paper tackles the problem of team coordination in multi-agent reinforcement learning by introducing the interactive world latent (IWoL) framework, which constructs a learnable representation space to capture inter-agent relations and task-specific information, achieving improved performance across four challenging benchmarks.
This work presents a novel representation learning framework, interactive world latent (IWoL), to facilitate team coordination in multi-agent reinforcement learning (MARL). Building effective representation for team coordination is a challenging problem, due to the intricate dynamics emerging from multi-agent interaction and incomplete information induced by local observations. Our key insight is to construct a learnable representation space that jointly captures inter-agent relations and task-specific world information by directly modeling communication protocols. This representation, we maintain fully decentralized execution with implicit coordination, all while avoiding the inherent drawbacks of explicit message passing, e.g., slower decision-making, vulnerability to malicious attackers, and sensitivity to bandwidth constraints. In practice, our representation can be used not only as an implicit latent for each agent, but also as an explicit message for communication. Across four challenging MARL benchmarks, we evaluate both variants and show that IWoL provides a simple yet powerful key for team coordination. Moreover, we demonstrate that our representation can be combined with existing MARL algorithms to further enhance their performance.