Stochastic Nonlinear Control via Finite-dimensional Spectral Dynamic Embedding
This work addresses control problems for stochastic systems, offering a novel method with theoretical guarantees, but it appears incremental as it builds on kernel approximation techniques.
The paper tackles optimal control for nonlinear stochastic systems by proposing Spectral Dynamics Embedding Control (SDEC), which uses finite-dimensional approximations to enable linear representation of value functions, and it shows favorable performance against existing algorithms on benchmark problems.
This paper proposes an approach, Spectral Dynamics Embedding Control (SDEC), to optimal control for nonlinear stochastic systems. This method reveals an infinite-dimensional feature representation induced by the system's nonlinear stochastic dynamics, enabling a linear representation of the state-action value function. For practical implementation, this representation is approximated using finite-dimensional truncations, specifically via two prominent kernel approximation methods: random feature truncation and Nystrom approximation. To characterize the effectiveness of these approximations, we provide an in-depth theoretical analysis to characterize the approximation error arising from the finite-dimension truncation and statistical error due to finite-sample approximation in both policy evaluation and policy optimization. Empirically, our algorithm performs favorably against existing stochastic control algorithms on several benchmark problems.