Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQG
This work addresses online control optimization with physical constraints for applications like robotics or systems engineering, representing an incremental advancement in constrained online LQG methods.
The paper tackles the online linear quadratic Gaussian (LQG) problem with linear constraints on stabilizing controllers, proposing the online Newton on manifold (ONM) algorithm to generate controllers adaptively. It establishes a regret bound based on the path-length of a benchmark sequence and validates the approach through simulations.
Recent advancement in online optimization and control has provided novel tools to study online linear quadratic regulator (LQR) problems, where cost matrices are time-varying and unknown in advance. In this work, we study the online linear quadratic Gaussian (LQG) problem over the manifold of stabilizing controllers that are linearly constrained to impose physical conditions such as sparsity. By adopting a Riemannian perspective, we propose the online Newton on manifold (ONM) algorithm, which generates an online controller on-the-fly based on the second-order information of the cost function sequence. To quantify the algorithm performance, we use the notion of regret, defined as the sub-optimality of the algorithm cumulative cost against a (locally) minimizing controller sequence. We establish a regret bound in terms of the path-length of the benchmark minimizer sequence, and we further verify the effectiveness of ONM via simulations.