Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
This work addresses the computational bottleneck in multi-agent game theory for researchers and practitioners, offering a scalable solution for large-scale stochastic games, though it is incremental as it builds on existing fictitious play and deep BSDE techniques.
The authors tackled the problem of finding Markovian Nash equilibrium in large N-player stochastic differential games, which is computationally challenging due to the curse of dimensionality. They proposed a deep neural network-based algorithm using fictitious play and deep BSDE methods, achieving accurate results for up to fifty-player games with common noise, a scale difficult for conventional methods.
We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large $N$-player stochastic differential games. Following the idea of fictitious play, we recast the $N$-player game into $N$ decoupled decision problems (one for each player) and solve them iteratively. The individual decision problem is characterized by a semilinear Hamilton-Jacobi-Bellman equation, to solve which we employ the recently developed deep BSDE method. The resulted algorithm can solve large $N$-player games for which conventional numerical methods would suffer from the curse of dimensionality. Multiple numerical examples involving identical or heterogeneous agents, with risk-neutral or risk-sensitive objectives, are tested to validate the accuracy of the proposed algorithm in large group games. Even for a fifty-player game with the presence of common noise, the proposed algorithm still finds the approximate Nash equilibrium accurately, which, to our best knowledge, is difficult to achieve by other numerical algorithms.