Learning to Cooperate and Communicate Over Imperfect Channels
This addresses the challenge of robust communication in decentralized multi-agent systems, though it is incremental as it builds on existing Q-learning methods.
The paper tackles the problem of multi-agent cooperation over unreliable communication channels by proposing a novel approach based on independent Q-learning, which allows agents to dynamically adapt message sizes and encode/decode messages, resulting in improved performance in a cooperative digit-prediction environment.
Information exchange in multi-agent systems improves the cooperation among agents, especially in partially observable settings. In the real world, communication is often carried out over imperfect channels. This requires agents to handle uncertainty due to potential information loss. In this paper, we consider a cooperative multi-agent system where the agents act and exchange information in a decentralized manner using a limited and unreliable channel. To cope with such channel constraints, we propose a novel communication approach based on independent Q-learning. Our method allows agents to dynamically adapt how much information to share by sending messages of different sizes, depending on their local observations and the channel's properties. In addition to this message size selection, agents learn to encode and decode messages to improve their jointly trained policies. We show that our approach outperforms approaches without adaptive capabilities in a novel cooperative digit-prediction environment and discuss its limitations in the traffic junction environment.