Unifying Hamilton-Jacobi Reachability and Reinforcement Learning
For safety-critical control, this provides a theoretical bridge between reachability analysis and deep RL, enabling safety guarantees in RL-based controllers.
The paper unifies Hamilton-Jacobi reachability and reinforcement learning by proving that a travel-cost value function solves a specific HJB PDE and that RL value iteration converges to its viscosity solution, with experiments validating convergence and safety semantics.
We unify Hamilton-Jacobi (HJ) reachability and Reinforcement Learning (RL) through a proposed running cost formulation. We prove that the resultant travel-cost value function is the unique bounded viscosity solution of a time-dependent Hamilton-Jacobi Bellman (HJB) Partial Differential Equation (PDE) with zero terminal data, whose negative sublevel set equals the strict backward-reachable tube. Using a forward reparameterization and a contraction inducing Bellman update, we show that fixed points of small-step RL value iteration converge to the viscosity solution of the forward discounted HJB. Experiments on a classical benchmark validate this connection by demonstrating convergence of learned value functions toward semi-Lagrangian HJB solutions and by quantifying approximation error across the state space. These results empirically support the theoretical analysis, showing that the proposed framework preserves reachability-based safety semantics while remaining compatible with deep RL implementations.