A Hierarchical Surrogate Model for Efficient Multi-Task Parameter Learning in Closed-Loop Control
This work addresses data efficiency and adaptability challenges for engineers tuning controllers in sequential decision-making scenarios, though it is incremental as it builds on existing Bayesian optimization techniques.
The authors tackled the problem of inefficient controller parameter tuning across multiple closed-loop control tasks by proposing a hierarchical Bayesian optimization framework that leverages structural knowledge of the system, resulting in improved sample efficiency and adaptability compared to black-box methods, as demonstrated in simulations.
Many control problems require repeated tuning and adaptation of controllers across distinct closed-loop tasks, where data efficiency and adaptability are critical. We propose a hierarchical Bayesian optimization (BO) framework that is tailored to efficient controller parameter learning in sequential decision-making and control scenarios for distinct tasks. Instead of treating the closed-loop cost as a black-box, our method exploits structural knowledge of the underlying problem, consisting of a dynamical system, a control law, and an associated closed-loop cost function. We construct a hierarchical surrogate model using Gaussian processes that capture the closed-loop state evolution under different parameterizations, while the task-specific weighting and accumulation into the closed-loop cost are computed exactly via known closed-form expressions. This allows knowledge transfer and enhanced data efficiency between different closed-loop tasks. The proposed framework retains sublinear regret guarantees on par with standard black-box BO, while enabling multi-task or transfer learning. Simulation experiments with model predictive control demonstrate substantial benefits in both sample efficiency and adaptability when compared to purely black-box BO approaches.