Practical Reinforcement Learning of Stabilizing Economic MPC
For control engineers dealing with nonlinear systems where accurate models are hard to obtain, this work offers a practical method to combine RL and MPC.
The paper proposes an improved reinforcement learning algorithm for tuning model predictive control, addressing the challenge of model inaccuracy. Simulations on a challenging example demonstrate the effectiveness of the approach.
Reinforcement Learning (RL) has demonstrated a huge potential in learning optimal policies without any prior knowledge of the process to be controlled. Model Predictive Control (MPC) is a popular control technique which is able to deal with nonlinear dynamics and state and input constraints. The main drawback of MPC is the need of identifying an accurate model, which in many cases cannot be easily obtained. Because of model inaccuracy, MPC can fail at delivering satisfactory closed-loop performance. Using RL to tune the MPC formulation or, conversely, using MPC as a function approximator in RL allows one to combine the advantages of the two techniques. This approach has important advantages, but it requires an adaptation of the existing algorithms. We therefore propose an improved RL algorithm for MPC and test it in simulations on a rather challenging example.