Safe Deep Reinforcement Learning for Spacecraft Reorientation with Pointing Keep-Out Constraint
For spacecraft control engineers, this work provides a method to enforce safety constraints in DRL-based reorientation, but it is incremental as it combines existing techniques (SAC, CBF) for a specific application.
This paper implements deep reinforcement learning with a safety filter for spacecraft reorientation under a pointing keep-out constraint, demonstrating that reward shaping alone cannot guarantee safety, while a CBF-based safety filter ensures constraint compliance.
This paper implements deep reinforcement learning (DRL) with a safety filter for spacecraft reorientation control with a single pointing keep-out zone. A new state space representation is designed which includes a compact representation of the attitude constraint zone. A reward function is formulated to achieve the control objective while enforcing the attitude constraint. The soft actor-critic (SAC) algorithm is adopted to handle continuous state and action space. A curriculum learning approach is implemented for agent training. To guarantee the compliance of the attitude constraint, a control barrier function (CBF)-based safety filter is implemented for agent deployment. Simulation results demonstrate the effectiveness of the proposed state space presentation and the designed reward function. Monte Carlo simulations underscore that reward shaping alone cannot guarantee the safety during reorientation maneuver. In contrast, with the CBF-based safety filter, the constraint can be guaranteed during maneuvers.