Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey
This is an incremental survey that organizes existing research for researchers in decentralized multi-agent systems.
The paper surveys fully decentralized cooperative multi-agent reinforcement learning methods, addressing the challenge of training agents without information about others, and reviews approaches for maximizing shared or individual rewards while discussing open questions.
Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks, but restrictions of real-world applications may require training the agents in a fully decentralized manner. Due to the lack of information about other agents, it is challenging to derive algorithms that can converge to the optimal joint policy in a fully decentralized setting. Thus, this research area has not been thoroughly studied. In this paper, we seek to systematically review the fully decentralized methods in two settings: maximizing a shared reward of all agents and maximizing the sum of individual rewards of all agents, and discuss open questions and future research directions.