Learning over Forward-Invariant Policy Classes: Reinforcement Learning without Safety Concerns
This addresses safety concerns in reinforcement learning for nonlinear systems like quadcopters, though it appears incremental by building on existing safe RL methods.
The paper tackles safe reinforcement learning by embedding safety into action representations using forward-invariance-induced design, resulting in improved closed-loop performance and switching efficiency for a quadcopter hover-regulation problem under disturbance, with all policies remaining safety-preserving.
This paper proposes a safe reinforcement learning (RL) framework based on forward-invariance-induced action-space design. The control problem is cast as a Markov decision process, but instead of relying on runtime shielding or penalty-based constraints, safety is embedded directly into the action representation. Specifically, we construct a finite admissible action set in which each discrete action corresponds to a stabilizing feedback law that preserves forward invariance of a prescribed safe state set. Consequently, the RL agent optimizes policies over a safe-by-construction policy class. We validate the framework on a quadcopter hover-regulation problem under disturbance. Simulation results show that the learned policy improves closed-loop performance and switching efficiency, while all evaluated policies remain safety-preserving. The proposed formulation decouples safety assurance from performance optimization and provides a promising foundation for safe learning in nonlinear systems.