Algorithms in Multi-Agent Systems: A Holistic Perspective from Reinforcement Learning and Game Theory
It provides a comprehensive overview for researchers in AI and multi-agent systems, but is incremental as it synthesizes existing methods rather than introducing new ones.
This survey addresses the challenges in multi-agent systems by integrating reinforcement learning with game theory, highlighting how solution concepts, fictitious self-play, and counterfactual regret minimization enhance algorithm performance in these scenarios.
Deep reinforcement learning (RL) has achieved outstanding results in recent years, which has led a dramatic increase in the number of methods and applications. Recent works are exploring learning beyond single-agent scenarios and considering multi-agent scenarios. However, they are faced with lots of challenges and are seeking for help from traditional game-theoretic algorithms, which, in turn, show bright application promise combined with modern algorithms and boosting computing power. In this survey, we first introduce basic concepts and algorithms in single agent RL and multi-agent systems; then, we summarize the related algorithms from three aspects. Solution concepts from game theory give inspiration to algorithms which try to evaluate the agents or find better solutions in multi-agent systems. Fictitious self-play becomes popular and has a great impact on the algorithm of multi-agent reinforcement learning. Counterfactual regret minimization is an important tool to solve games with incomplete information, and has shown great strength when combined with deep learning.