Markov-Nash Equilibria in Mean-Field Games with Discounted Cost
For researchers in mean-field game theory, this provides a rigorous existence result under mild assumptions, extending prior work to more general state spaces and cost criteria.
This paper proves existence of mean-field equilibrium for discrete-time mean-field games with discounted cost and locally compact state spaces, and shows that the mean-field equilibrium policy is approximately Markov-Nash for large finite populations.
In this paper, we consider discrete-time dynamic games of the mean-field type with a finite number $N$ of agents subject to an infinite-horizon discounted-cost optimality criterion. The state space of each agent is a locally compact Polish space. At each time, the agents are coupled through the empirical distribution of their states, which affects both the agents' individual costs and their state transition probabilities. We introduce a new solution concept of the Markov-Nash equilibrium, under which a policy is player-by-player optimal in the class of all Markov policies. Under mild assumptions, we demonstrate the existence of a mean-field equilibrium in the infinite-population limit $N \to \infty$, and then show that the policy obtained from the mean-field equilibrium is approximately Markov-Nash when the number of agents $N$ is sufficiently large.