A predictive safety filter for learning-based control of constrained nonlinear dynamical systems
This addresses safety concerns in deploying RL for real-world applications like robotics or autonomous systems, though it is an incremental improvement building on existing model predictive control methods.
The paper tackles the challenge of applying reinforcement learning to real-world systems with safety constraints by introducing a predictive safety filter that ensures safe control inputs for nonlinear dynamical systems, enabling any RL algorithm to be used without modifications.
The transfer of reinforcement learning (RL) techniques into real-world applications is challenged by safety requirements in the presence of physical limitations. Most RL methods, in particular the most popular algorithms, do not support explicit consideration of state and input constraints. In this paper, we address this problem for nonlinear systems with continuous state and input spaces by introducing a predictive safety filter, which is able to turn a constrained dynamical system into an unconstrained safe system and to which any RL algorithm can be applied `out-of-the-box'. The predictive safety filter receives the proposed control input and decides, based on the current system state, if it can be safely applied to the real system, or if it has to be modified otherwise. Safety is thereby established by a continuously updated safety policy, which is based on a model predictive control formulation using a data-driven system model and considering state and input dependent uncertainties.