Online estimation and control with optimal pathlength regret
This work addresses the challenge of adapting online learning to dynamic environments for applications in control systems, though it is incremental as it extends existing pathlength regret concepts to new domains.
The paper tackles the problem of online control and estimation in non-stationary linear dynamical systems by deriving the first pathlength regret bounds, which link algorithm performance to temporal variation in inputs, and shows through simulations that their algorithms outperform traditional H2 and H∞ methods in time-varying environments.
A natural goal when designing online learning algorithms for non-stationary environments is to bound the regret of the algorithm in terms of the temporal variation of the input sequence. Intuitively, when the variation is small, it should be easier for the algorithm to achieve low regret, since past observations are predictive of future inputs. Such data-dependent "pathlength" regret bounds have recently been obtained for a wide variety of online learning problems, including OCO and bandits. We obtain the first pathlength regret bounds for online control and estimation (e.g. Kalman filtering) in linear dynamical systems. The key idea in our derivation is to reduce pathlength-optimal filtering and control to certain variational problems in robust estimation and control; these reductions may be of independent interest. Numerical simulations confirm that our pathlength-optimal algorithms outperform traditional $H_2$ and $H_{\infty}$ algorithms when the environment varies over time.