Reward-Independent Messaging for Decentralized Multi-Agent Reinforcement Learning
This addresses the challenge of coordination in complex, decentralized environments for multi-agent systems, though it is incremental as it builds on existing emergent communication research.
The paper tackled the problem of enabling effective communication among decentralized, independent agents in multi-agent reinforcement learning (MARL) without assuming cooperation or direct rewards, and the result was that their MARL-CPC framework, with algorithms like Bandit-CPC and IPPO-CPC, outperformed standard message-as-action approaches in non-cooperative tasks.
In multi-agent reinforcement learning (MARL), effective communication improves agent performance, particularly under partial observability. We propose MARL-CPC, a framework that enables communication among fully decentralized, independent agents without parameter sharing. MARL-CPC incorporates a message learning model based on collective predictive coding (CPC) from emergent communication research. Unlike conventional methods that treat messages as part of the action space and assume cooperation, MARL-CPC links messages to state inference, supporting communication in non-cooperative, reward-independent settings. We introduce two algorithms -Bandit-CPC and IPPO-CPC- and evaluate them in non-cooperative MARL tasks. Benchmarks show that both outperform standard message-as-action approaches, establishing effective communication even when messages offer no direct benefit to the sender. These results highlight MARL-CPC's potential for enabling coordination in complex, decentralized environments.