Optimal Control of Nonlinear Systems with Unknown Dynamics
This addresses the challenge of optimal control in real-world systems where dynamics are often unknown, offering a practical solution for engineers and researchers, though it appears incremental as it builds on existing actor-critic and Koopman operator frameworks.
The paper tackles the problem of finding optimal controllers for nonlinear systems with unknown dynamics by proposing a data-driven method that integrates the Koopman operator with actor-critic concepts to estimate gradients and tune policy parameters via gradient descent, achieving improved control performance in simulations compared to classical methods that assume known dynamics.
This paper presents a data-driven method to find a closed-loop optimal controller, which minimizes a specified infinite-horizon cost function for systems with unknown dynamics. Suppose the closed-loop optimal controller can be parameterized by a given class of functions, hereafter referred to as the policy. The proposed method introduces a novel gradient estimation framework, which approximates the gradient of the cost function with respect to the policy parameters via integrating the Koopman operator with the classical concept of actor-critic. This enables the policy parameters to be tuned iteratively using gradient descent to achieve an optimal controller, leveraging the linearity of the Koopman operator. The convergence analysis of the proposed framework is provided. The control performance of the proposed method is evaluated through simulations compared with classical optimal control methods that usually assume the dynamics are known.