Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry
This work addresses the need for more adaptive and robust attitude control in spacecraft re-entry, a safety-critical aerospace application, but the improvements are incremental over strong baselines.
Deep reinforcement learning and hybrid controllers achieve superior performance over traditional PID controllers for spacecraft attitude control during atmospheric re-entry, with better angle-of-attack tracking and robustness to variations in mass, inertia, and actuator bandwidth.
Deep reinforcement learning has the potential to solve attitude control problems more adaptively, precisely, and robustly by handling nonlinear dynamics, uncertainties, and failure cases more effectively than traditional attitude control approaches. We explore reinforcement learning (RL) for attitude control in spacecraft re-entry. An industry-standard proportional-integral-derivative controller with gain scheduling serves as a strong baseline for model-free RL and hybrid controllers that combine these two approaches. We formalize the application in the RL framework to apply continuous, off-policy RL. State-of-the-art RL achieves comparable performance to traditional control approaches in this domain. However, its out-of-distribution generalization is not sufficient. Hence, we use dynamics randomization to introduce challenging task variations during training and enforce generalization in a predefined operational envelope. Finally, we assess the best obtained RL-based controllers with application-specific metrics to show superior performance in comparison to traditional controllers in the operational envelope, that is, hybrid controllers are able to track the angle of attack better and are more robust under variations of mass, inertia tensor, and flap actuator bandwidth.