Low Entropy Communication in Multi-Agent Reinforcement Learning
This addresses the problem of communication inefficiency for multi-agent systems in real-world applications, representing an incremental improvement.
The paper tackled the problem of high message entropy in multi-agent reinforcement learning communication, which can limit efficiency in resource-constrained scenarios, and achieved a result of reducing message entropy by up to 90% with minimal loss in cooperation performance.
Communication in multi-agent reinforcement learning has been drawing attention recently for its significant role in cooperation. However, multi-agent systems may suffer from limitations on communication resources and thus need efficient communication techniques in real-world scenarios. According to the Shannon-Hartley theorem, messages to be transmitted reliably in worse channels require lower entropy. Therefore, we aim to reduce message entropy in multi-agent communication. A fundamental challenge is that the gradients of entropy are either 0 or infinity, disabling gradient-based methods. To handle it, we propose a pseudo gradient descent scheme, which reduces entropy by adjusting the distributions of messages wisely. We conduct experiments on two base communication frameworks with six environment settings and find that our scheme can reduce message entropy by up to 90% with nearly no loss of cooperation performance.