Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System
Provides a method to enforce safety constraints in DRL for industrial cyber-physical systems without complex reward shaping, but is incremental and limited to simulated 1-DoF systems.
Embedded a differentiable physics model into the PPO actor loss to penalize anticipated safety violations, reducing constraint violations while maintaining tracking performance on a 1-DoF helicopter simulation.
Deep reinforcement learning (DRL) offers powerful control for industrial cyber-physical systems (ICPSs), but its "black-box" exploration risks violating strict hardware safety limits. Typically, these constraints are managed through complex reward shaping. In this work-in-progress paper, we embed a differentiable physics model directly into the proximal policy optimization (PPO) actor loss function. By simulating short-horizon future trajectories during training, the policy is penalized for anticipated safety violations independent of the task-reward signal. Evaluated on a simulated 1-degree-of-freedom helicopter testbed with strict pitch constraints, our physics-informed soft regularizations substantially reduce constraint violations while maintaining reliable target tracking.