Data-Driven Stochastic Optimal Control in Reproducing Kernel Hilbert Spaces
This provides a data-driven method for nonlinear stochastic optimal control, which is incremental as it builds on existing operator regression techniques to handle unknown systems.
The paper tackles the problem of optimal control for nonlinear stochastic systems with unknown dynamics and cost functions by embedding state densities into a reproducing kernel Hilbert space and using operator regression to learn Markov transition operators. It demonstrates the approach on tasks like depth regulation of an autonomous underwater vehicle, achieving scalable solutions with linear complexity in state dimensionality.
This paper proposes a fully data-driven approach for optimal control of nonlinear control-affine systems represented by a stochastic diffusion. The focus is on the scenario where both the nonlinear dynamics and stage cost functions are unknown, while only a control penalty function and constraints are provided. To this end, we embed state probability densities into a reproducing kernel Hilbert space (RKHS) to leverage recent advances in operator regression, thereby identifying Markov transition operators associated with controlled diffusion processes. This operator learning approach integrates naturally with convex operator-theoretic Hamilton-Jacobi-Bellman recursions that scale linearly with state dimensionality, effectively solving a wide range of nonlinear optimal control problems. Numerical results demonstrate its ability to address diverse nonlinear control tasks, including the depth regulation of an autonomous underwater vehicle.