On the System Theoretic Offline Learning of Continuous-Time LQR with Exogenous Disturbances
This work addresses offline control design for LQR systems with disturbances, which is incremental as it builds on existing adaptive dynamic programming methods.
The paper tackles offline learning of continuous-time linear quadratic regulator (LQR) strategies with uncertain disturbances, developing algorithms based on adaptive dynamic programming and Lyapunov analysis to handle scenarios with estimable and unknown disturbances, and provides stability guarantees and numerical validations.
We analyze offline designs of linear quadratic regulator (LQR) strategies with uncertain disturbances. First, we consider the scenario where the exogenous variable can be estimated in a controlled environment, and subsequently, consider a more practical and challenging scenario where it is unknown in a stochastic setting. Our approach builds on the fundamental learning-based framework of adaptive dynamic programming (ADP), combined with a Lyapunov-based analytical methodology to design the algorithms and derive sample-based approximations motivated from the Markov decision process (MDP)-based approaches. For the scenario involving non-measurable disturbances, we further establish stability and convergence guarantees for the learned control gains under sample-based approximations. The overall methodology emphasizes simplicity while providing rigorous guarantees. Finally, numerical experiments focus on the intricacies and validations for the design of offline continuous-time LQR with exogenous disturbances.