Learning Stabilizing Controllers for Unstable Linear Quadratic Regulators from a Single Trajectory
This addresses the challenge of stabilizing unknown dynamical systems in control theory, offering a practical solution for initializing algorithms that require a stabilizing controller, though it appears incremental with heuristic improvements.
The paper tackles the problem of learning stabilizing controllers for unknown linear quadratic regulators from a single trajectory, presenting robust SDP-based methods and an efficient algorithm called eXploration that quickly identifies a stabilizing controller with high probability, adding only constant regret to existing algorithms.
The principal task to control dynamical systems is to ensure their stability. When the system is unknown, robust approaches are promising since they aim to stabilize a large set of plausible systems simultaneously. We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR). We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set. We further show that the feasibility conditions of the proposed SDPs are \emph{equivalent}. Using the derived robust controller syntheses, we propose an efficient data dependent algorithm -- \textsc{eXploration} -- that with high probability quickly identifies a stabilizing controller. Our approach can be used to initialize existing algorithms that require a stabilizing controller as an input while adding constant to the regret. We further propose different heuristics which empirically reduce the number of steps taken by \textsc{eXploration} and reduce the suffered cost while searching for a stabilizing controller.