Deep QP Safety Filter: Model-free Learning for Reachability-based Safety Filter
For practitioners of safe reinforcement learning, this provides a principled, model-free safety filter that works on black-box systems without requiring system dynamics knowledge.
This paper introduces Deep QP Safety Filter, a model-free safety layer for black-box dynamical systems that combines HJ reachability with learned neural networks. It reduces pre-convergence failures and accelerates RL learning across diverse systems, including hybrid systems, outperforming strong baselines.
We introduce Deep QP Safety Filter, a fully data-driven safety layer for black-box dynamical systems. Our method learns a Quadratic-Program (QP) safety filter without model knowledge by combining Hamilton-Jacobi (HJ) reachability with model-free learning. We construct contraction-based losses for both the safety value and its derivatives, and train two neural networks accordingly. In the exact setting, the learned critic converges to the viscosity solution (and its derivative), even for non-smooth values. Across diverse dynamical systems -- even including a hybrid system -- and multiple RL tasks, Deep QP Safety Filter substantially reduces pre-convergence failures while accelerating learning toward higher returns than strong baselines, offering a principled and practical route to safe, model-free control.