Deep Forward-Backward SDEs for Min-max Control
This addresses control problems in stochastic environments, offering a novel computational method for differential games, though it appears incremental as an extension of existing SDE and neural network techniques to this domain.
The paper tackles solving stochastic differential games for nonlinear systems by representing the Hamilton-Jacobi-Isaacs PDE as forward-backward SDEs and solving them numerically with importance sampling and an LSTM network. The algorithm was tested on two simulated examples and compared to standard risk-neutral control, showing improved performance with specific numerical gains reported.
This paper presents a novel approach to numerically solve stochastic differential games for nonlinear systems. The proposed approach relies on the nonlinear Feynman-Kac theorem that establishes a connection between parabolic deterministic partial differential equations and forward-backward stochastic differential equations. Using this theorem the Hamilton-Jacobi-Isaacs partial differential equation associated with differential games is represented by a system of forward-backward stochastic differential equations. Numerical solution of the aforementioned system of stochastic differential equations is performed using importance sampling and a Long-Short Term Memory recurrent neural network, which is trained in an offline fashion. The resulting algorithm is tested on two example systems in simulation and compared against the standard risk neutral stochastic optimal control formulations.