Deep Fictitious Play for Stochastic Differential Games
This work addresses the computational challenge of solving complex stochastic differential games in fields like economics and finance, representing an incremental advancement by applying deep learning to an existing game theory framework.
The paper tackles the problem of computing Nash equilibria for asymmetric N-player non-zero-sum stochastic differential games by developing a deep learning algorithm called deep fictitious play, which decomposes the game into N decoupled stochastic control problems approximated by neural networks. The algorithm is shown to be scalable, parallel, and model-free, with convergence proven under appropriate assumptions and performance validated against closed-form solutions for linear quadratic games.
In this paper, we apply the idea of fictitious play to design deep neural networks (DNNs), and develop deep learning theory and algorithms for computing the Nash equilibrium of asymmetric $N$-player non-zero-sum stochastic differential games, for which we refer as \emph{deep fictitious play}, a multi-stage learning process. Specifically at each stage, we propose the strategy of letting individual player optimize her own payoff subject to the other players' previous actions, equivalent to solve $N$ decoupled stochastic control optimization problems, which are approximated by DNNs. Therefore, the fictitious play strategy leads to a structure consisting of $N$ DNNs, which only communicate at the end of each stage. The resulted deep learning algorithm based on fictitious play is scalable, parallel and model-free, {\it i.e.}, using GPU parallelization, it can be applied to any $N$-player stochastic differential game with different symmetries and heterogeneities ({\it e.g.}, existence of major players). We illustrate the performance of the deep learning algorithm by comparing to the closed-form solution of the linear quadratic game. Moreover, we prove the convergence of fictitious play under appropriate assumptions, and verify that the convergent limit forms an open-loop Nash equilibrium. We also discuss the extensions to other strategies designed upon fictitious play and closed-loop Nash equilibrium in the end.