Thompson Sampling-Based Learning and Control for Unknown Dynamic Systems
This work addresses the problem of designing controllers for unknown dynamic systems, offering a novel approach that is incremental but extends applicability beyond finite parametric representations.
The paper tackles the limitation of Thompson sampling in control design by proposing a parameterization method using reproducing kernel Hilbert spaces, enabling control law learning in general function spaces without structural restrictions, and demonstrates exponential learning rates and bounded regret in numerical experiments on nonlinear systems.
Thompson sampling (TS) is an effective method to explore parametric uncertainties and can therefore be used for active learning-based controller design. However, TS relies on finite parametric representations, which limits its applicability to more general spaces, which are more commonly encountered in control system design. To address this issue, this work pro poses a parameterization method for control law learning using reproducing kernel Hilbert spaces and designs a data-driven active learning control approach. Specifically, the proposed method treats the control law as an element in a function space, allowing the design of control laws without imposing restrictions on the system structure or the form of the controller. A TS framework is proposed in this work to explore potential optimal control laws, and the convergence guarantees are further provided for the learning process. Theoretical analysis shows that the proposed method learns the relationship between control laws and closed-loop performance metrics at an exponential rate, and the upper bound of control regret is also derived. Numerical experiments on controlling unknown nonlinear systems validate the effectiveness of the proposed method.