Leilei Cui

Leilei Cui, Richard D. Braatz

A gradient-based method is proposed for solving the linear quadratic regulator (LQR) problem for linear systems with nonlinear dependence on time-invariant probabilistic parametric uncertainties. The approach explicitly accounts for model uncertainty and ensures robust performance. By leveraging polynomial chaos theory (PCT) in conjunction with policy optimization techniques, the original stochastic system is lifted into a high-dimensional linear time-invariant (LTI) system with structured state-feedback control. A first-order gradient descent algorithm is then developed to directly optimize the structured feedback gain and iteratively minimize the LQR cost. We rigorously establish linear convergence of the gradient descent algorithm and show that the PCT-based approximation error decays algebraically at a rate $O(N^{-p})$ for any positive integer $p$, where $N$ denotes the order of the polynomials. Numerical examples demonstrate that the proposed method achieves significantly higher computational efficiency than conventional bilinear matrix inequality (BMI)-based approaches.

Arthur Castello B. de Oliveira, Leilei Cui, Eduardo D. Sontag

This work explores generalizations of the Polyak-Lojasiewicz inequality (PLI) and their implications for the convergence behavior of gradient flows in optimization problems. Motivated by the continuous-time linear quadratic regulator (CT-LQR) policy optimization problem -- where only a weaker version of the PLI is characterized in the literature -- this work shows that while weaker conditions are sufficient for global convergence to, and optimality of the set of critical points of the cost function, the "profile" of the gradient flow solution can change significantly depending on which "flavor" of inequality the cost satisfies. After a general theoretical analysis, we focus on fitting the CT-LQR policy optimization problem to the proposed framework, showing that, in fact, it can never satisfy a PLI in its strongest form. We follow up our analysis with a brief discussion on the difference between continuous- and discrete-time LQR policy optimization, and end the paper with some intuition on the extension of this framework to optimization problems with L1 regularization and solved through proximal gradient flows.