Finite Time Regret Bounds for Minimum Variance Control of Autoregressive Systems with Exogenous Inputs
This addresses the challenge of acceptable transients in adaptive control for industrial applications, though it is incremental as it builds on prior work with specific algorithmic improvements.
The paper tackles the problem of poor initial performance in adaptive minimum variance controllers for ARX systems by proposing PIECE, a modified certainty equivalence controller with probing inputs, achieving a C log T regret bound for bounded noise and C log^2 T for sub-Gaussian noise.
Minimum variance controllers have been employed in a wide-range of industrial applications. A key challenge experienced by many adaptive controllers is their poor empirical performance in the initial stages of learning. In this paper, we address the problem of initializing them so that they provide acceptable transients, and also provide an accompanying finite-time regret analysis, for adaptive minimum variance control of an auto-regressive system with exogenous inputs (ARX). Following [3], we consider a modified version of the Certainty Equivalence (CE) adaptive controller, which we call PIECE, that utilizes probing inputs for exploration. We show that it has a $C \log T$ bound on the regret after $T$ time-steps for bounded noise, and $C\log^2 T$ in the case of sub-Gaussian noise. The simulation results demonstrate the advantage of PIECE over the algorithm proposed in [3] as well as the standard Certainty Equivalence controller especially in the initial learning phase. To the best of our knowledge, this is the first work that provides finite-time regret bounds for an adaptive minimum variance controller.