Physics-Regulated Deep Reinforcement Learning: Invariant Embeddings
This work addresses safety-critical issues in autonomous systems, offering an incremental improvement by combining existing physics models with deep reinforcement learning.
The paper tackles the problem of ensuring safety in autonomous systems by proposing Phy-DRL, a physics-regulated deep reinforcement learning framework that integrates physics models with data-driven methods, resulting in guaranteed safety, fewer learning parameters, and faster training compared to purely data-driven or model-based approaches.
This paper proposes the Phy-DRL: a physics-regulated deep reinforcement learning (DRL) framework for safety-critical autonomous systems. The Phy-DRL has three distinguished invariant-embedding designs: i) residual action policy (i.e., integrating data-driven-DRL action policy and physics-model-based action policy), ii) automatically constructed safety-embedded reward, and iii) physics-model-guided neural network (NN) editing, including link editing and activation editing. Theoretically, the Phy-DRL exhibits 1) a mathematically provable safety guarantee and 2) strict compliance of critic and actor networks with physics knowledge about the action-value function and action policy. Finally, we evaluate the Phy-DRL on a cart-pole system and a quadruped robot. The experiments validate our theoretical results and demonstrate that Phy-DRL features guaranteed safety compared to purely data-driven DRL and solely model-based design while offering remarkably fewer learning parameters and fast training towards safety guarantee.