Regret-Guaranteed Safe Switching: LQR Setting with Unknown Dynamics
For control systems requiring safe switching under unknown dynamics, this work provides the first regret-guaranteed algorithm with explicit bounds.
The paper tackles safe switching control in LQR systems with unknown dynamics, introducing a notion of (α,β)-controllability to ensure bounded state norms. The proposed algorithm achieves an expected regret of O(|M|^{1/4} n_s^{3/4} + n_m) compared to an offline benchmark.
We consider learning-based control in LQR setting, where the parameters associated with each mode are a priori unknown. The next mode to be activated is revealed online only at the time of switching. The objective is to determine both the switching times and the control gains for each mode such that (1) the norm of the system state remains bounded according to a prescribed criterion, and (2) the accumulated cost is minimized. To formalize the state-norm requirement, we introduce the notion of $(α,β)$-controllability for given parameters $α$ and $β$. We first study the problem in a known model setting and show that, under the switching mechanism described above and under the assumption that each mode is visited infinitely often, the strategy that minimizes the average expected cost consists of applying, in each mode, the feedback gain obtained from the solution of the discrete algebraic Riccati equation, while selecting dwell times that sufficiently satisfy the controllability condition. We refer to this strategy as the benchmark policy. Next, we propose an algorithm for the unknown-model setting that minimizes the regret, defined as the difference between the cumulative cost incurred by the online algorithm and that of the offline benchmark. By accurately estimating dwell-time errors, our method achieves an expected regret of $\mathcal{O}(|\mathcal{M}|^{1/4} n_s^{3/4} + n_m)$, where $n_s$ denotes the number of switches, $|\mathcal{M}|$ is the number of modes, and $n_m$ is the number of malignant switches.