Value Approximation for Two-Player General-Sum Differential Games with State Constraints
This addresses the curse of dimensionality in differential games for robotics and control applications, but is incremental as it builds on existing PINN methods.
The paper tackled the challenge of solving Hamilton-Jacobi-Isaacs PDEs for two-player differential games with state constraints, where vanilla physics-informed neural networks fail due to discontinuous solutions, and found that a hybrid learning method outperformed alternatives in generalization and safety performance in simulations up to 13 dimensions.
Solving Hamilton-Jacobi-Isaacs (HJI) PDEs numerically enables equilibrial feedback control in two-player differential games, yet faces the curse of dimensionality (CoD). While physics-informed neural networks (PINNs) have shown promise in alleviating CoD in solving PDEs, vanilla PINNs fall short in learning discontinuous solutions due to their sampling nature, leading to poor safety performance of the resulting policies when values are discontinuous due to state or temporal logic constraints. In this study, we explore three potential solutions to this challenge: (1) a hybrid learning method that is guided by both supervisory equilibria and the HJI PDE, (2) a value-hardening method where a sequence of HJIs are solved with increasing Lipschitz constant on the constraint violation penalty, and (3) the epigraphical technique that lifts the value to a higher dimensional state space where it becomes continuous. Evaluations through 5D and 9D vehicle and 13D drone simulations reveal that the hybrid method outperforms others in terms of generalization and safety performance by taking advantage of both the supervisory equilibrium values and costates, and the low cost of PINN loss gradients.