Regret-optimal control in dynamic environments
This work addresses the challenge of control in dynamic environments where no single controller performs well over time, offering a novel regret-optimal approach for applications like robotics or autonomous systems, though it is incremental as it builds on prior regret minimization and control theory.
The paper tackles the problem of designing an online controller for linear time-varying systems that minimizes dynamic regret against the best sequence of control actions in hindsight, rather than static regret against a fixed controller. It derives the controller's state-space structure via a reduction to H∞ control and provides a tight data-dependent regret bound in terms of disturbance energy, with numerical experiments showing it interpolates between H2-optimal and H∞-optimal performance across environments.
We consider control in linear time-varying dynamical systems from the perspective of regret minimization. Unlike most prior work in this area, we focus on the problem of designing an online controller which minimizes regret against the best dynamic sequence of control actions selected in hindsight (dynamic regret), instead of the best fixed controller in some specific class of controllers (static regret). This formulation is attractive when the environment changes over time and no single controller achieves good performance over the entire time horizon. We derive the state-space structure of the regret-optimal controller via a novel reduction to $H_{\infty}$ control and present a tight data-dependent bound on its regret in terms of the energy of the disturbance. Our results easily extend to the model-predictive setting where the controller can anticipate future disturbances and to settings where the controller only affects the system dynamics after a fixed delay. We present numerical experiments which show that our regret-optimal controller interpolates between the performance of the $H_2$-optimal and $H_{\infty}$-optimal controllers across stochastic and adversarial environments.