Q-Learning in Regularized Mean-field Games
This work addresses the challenge of robust learning in mean-field games for applications in multi-agent systems, though it appears incremental as it builds on existing methods with regularization.
The paper tackles the problem of learning in regularized mean-field games under infinite-horizon discounted rewards by introducing a strongly concave regularization term to the reward function, resulting in a more robust reinforcement learning algorithm that enables error analysis without restrictive convexity assumptions.
In this paper, we introduce a regularized mean-field game and study learning of this game under an infinite-horizon discounted reward function. Regularization is introduced by adding a strongly concave regularization function to the one-stage reward function in the classical mean-field game model. We establish a value iteration based learning algorithm to this regularized mean-field game using fitted Q-learning. The regularization term in general makes reinforcement learning algorithm more robust to the system components. Moreover, it enables us to establish error analysis of the learning algorithm without imposing restrictive convexity assumptions on the system components, which are needed in the absence of a regularization term.