A modular framework for stabilizing deep reinforcement learning control
This work addresses stability issues in control systems for applications like industrial automation, though it appears incremental as it builds on existing parameterization and behavioral systems methods.
The authors tackled the problem of ensuring stability in deep reinforcement learning controllers by integrating the Youla-Kucera parameterization with a data-driven internal model, resulting in a framework that maintains stability guarantees while leveraging neural networks for nonlinear stable operators.
We propose a framework for the design of feedback controllers that combines the optimization-driven and model-free advantages of deep reinforcement learning with the stability guarantees provided by using the Youla-Kucera parameterization to define the search domain. Recent advances in behavioral systems allow us to construct a data-driven internal model; this enables an alternative realization of the Youla-Kucera parameterization based entirely on input-output exploration data. Using a neural network to express a parameterized set of nonlinear stable operators enables seamless integration with standard deep learning libraries. We demonstrate the approach on a realistic simulation of a two-tank system.