Neural Policy Iteration for Stochastic Optimal Control: A Physics-Informed Approach
This work addresses stochastic optimal control problems for applications like robotics and control systems, offering an incremental extension of existing methods with theoretical guarantees.
The authors tackled stochastic optimal control problems by proposing a physics-informed neural network policy iteration framework that extends deterministic approaches to stochastic settings, achieving global exponential convergence and demonstrating effectiveness on benchmark problems including high-dimensional linear quadratic regulation up to 10D.
We propose a physics-informed neural network policy iteration (PINN-PI) framework for solving stochastic optimal control problems governed by second-order Hamilton--Jacobi--Bellman (HJB) equations. At each iteration, a neural network is trained to approximate the value function by minimizing the residual of a linear PDE induced by a fixed policy. This linear structure enables systematic $L^2$ error control at each policy evaluation step, and allows us to derive explicit Lipschitz-type bounds that quantify how value gradient errors propagate to the policy updates. This interpretability provides a theoretical basis for evaluating policy quality during training. Our method extends recent deterministic PINN-based approaches to stochastic settings, inheriting the global exponential convergence guarantees of classical policy iteration under mild conditions. We demonstrate the effectiveness of our method on several benchmark problems, including stochastic cartpole, pendulum problems and high-dimensional linear quadratic regulation (LQR) problems in up to 10D.