Robust nonlinear set-point control with reinforcement learning
This work addresses robustness issues in reinforcement learning for nonlinear control, enabling direct deployment to real-world systems, though it is incremental in nature.
The paper tackled the challenge of applying reinforcement learning to nonlinear set-point control by proposing three ideas: using a prior feedback controller, integrated errors, and model ensembles, resulting in more efficient training and robust controllers that were validated on real-world and simulated systems.
There has recently been an increased interest in reinforcement learning for nonlinear control problems. However standard reinforcement learning algorithms can often struggle even on seemingly simple set-point control problems. This paper argues that three ideas can improve reinforcement learning methods even for highly nonlinear set-point control problems: 1) Make use of a prior feedback controller to aid amplitude exploration. 2) Use integrated errors. 3) Train on model ensembles. Together these ideas lead to more efficient training, and a trained set-point controller that is more robust to modelling errors and thus can be directly deployed to real-world nonlinear systems. The claim is supported by experiments with a real-world nonlinear cascaded tank process and a simulated strongly nonlinear pH-control system.