Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control
This addresses safety concerns for real-world robot control using RL, though it is incremental as it builds on existing safe motion planning methods.
The paper tackles the problem of ensuring safety during reinforcement learning training for robots in critical environments by proposing a Reachability-based Trajectory Safeguard (RTS) that uses reachability analysis to precompute safe trajectories and adjust unsafe choices, demonstrating efficacy on three nonlinear robot models including a 12-D quadrotor drone in simulation with comparisons to state-of-the-art methods.
Reinforcement Learning (RL) algorithms have achieved remarkable performance in decision making and control tasks due to their ability to reason about long-term, cumulative reward using trial and error. However, during RL training, applying this trial-and-error approach to real-world robots operating in safety critical environment may lead to collisions. To address this challenge, this paper proposes a Reachability-based Trajectory Safeguard (RTS), which leverages reachability analysis to ensure safety during training and operation. Given a known (but uncertain) model of a robot, RTS precomputes a Forward Reachable Set of the robot tracking a continuum of parameterized trajectories. At runtime, the RL agent selects from this continuum in a receding-horizon way to control the robot; the FRS is used to identify if the agent's choice is safe or not, and to adjust unsafe choices. The efficacy of this method is illustrated on three nonlinear robot models, including a 12-D quadrotor drone, in simulation and in comparison with state-of-the-art safe motion planning methods.