Dual Approaches to Stochastic Control via SPDEs and the Pathwise Hopf Formula
This work addresses the curse of dimensionality in stochastic control for applications such as high-dimensional PDEs and reinforcement learning, offering incremental improvements to existing dual formulations.
The paper tackles high-dimensional stochastic control problems by developing dual approaches that compute robust dual bounds, proving the generalized Hopf formula under mild conditions and demonstrating effectiveness in complementing primal methods like deep BSDE and actor-critic in numerical experiments.
We develop dual approaches for continuous-time stochastic control problems, enabling the computation of robust dual bounds in high-dimensional state and control spaces. Building on the dual formulation proposed in [L. C. G. Rogers, SIAM Journal on Control and Optimization, 46 (2007), pp. 1116--1132], we first formulate the inner optimization problem as a stochastic partial differential equation (SPDE); the expectation of its solution yields the dual bound. Curse-of-dimensionality-free methods are proposed based on the Pontryagin maximum principle and the generalized Hopf formula. In the process, we prove the generalized Hopf formula, first introduced as a conjecture in [Y. T. Chow, J. Darbon, S. Osher, and W. Yin, Journal of Computational Physics 387 (2019), pp. 376--409], under mild conditions. Numerical experiments demonstrate that our dual approaches effectively complement primal methods, including the deep BSDE method for solving high-dimensional PDEs and the deep actor-critic method in reinforcement learning.