PAC Reinforcement Learning Algorithm for General-Sum Markov Games
This work provides a theoretical framework for PAC algorithms in MARL, which is incremental as it builds on existing methods like Nash Q-learning.
The paper tackles the problem of multi-agent reinforcement learning in general-sum Markov games by extending Nash Q-learning with delayed Q-learning to create a new PAC algorithm, and demonstrates its performance and robustness through comparative numerical results.
This paper presents a theoretical framework for probably approximately correct (PAC) multi-agent reinforcement learning (MARL) algorithms for Markov games. The paper offers an extension to the well-known Nash Q-learning algorithm, using the idea of delayed Q-learning, in order to build a new PAC MARL algorithm for general-sum Markov games. In addition to guiding the design of a provably PAC MARL algorithm, the framework enables checking whether an arbitrary MARL algorithm is PAC. Comparative numerical results demonstrate performance and robustness.