Safe and Stable Closed-Loop Learning for Neural-Network-Supported Model Predictive Control
This work addresses safe and stable policy learning for control systems, particularly in applications like robotics or autonomous vehicles, but it is incremental as it builds on existing MPC and Bayesian optimization methods.
The paper tackles the challenge of safely learning control policies for predictive controllers with incomplete process information by using Bayesian optimization to tune neural-network-parametrized MPC stage costs, achieving rigorous probabilistic safety guarantees and improved closed-loop performance, as demonstrated in a numeric example.
Safe learning of control policies remains challenging, both in optimal control and reinforcement learning. In this article, we consider safe learning of parametrized predictive controllers that operate with incomplete information about the underlying process. To this end, we employ Bayesian optimization for learning the best parameters from closed-loop data. Our method focuses on the system's overall long-term performance in closed-loop while keeping it safe and stable. Specifically, we parametrize the stage cost function of an MPC using a feedforward neural network. This allows for a high degree of flexibility, enabling the system to achieve a better closed-loop performance with respect to a superordinate measure. However, this flexibility also necessitates safety measures, especially with respect to closed-loop stability. To this end, we explicitly incorporated stability information in the Bayesian-optimization-based learning procedure, thereby achieving rigorous probabilistic safety guarantees. The proposed approach is illustrated using a numeric example.