A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines
This work addresses the need for improved control in reusable rocket engines, offering a novel solution to a domain-specific bottleneck in aerospace engineering.
The paper tackles the problem of controlling transient phases in liquid rocket engines, which are traditionally managed with open-loop systems, by applying deep reinforcement learning to achieve optimal performance during start-up. The result shows that the learned policy outperforms tuned open-loop sequences and PID controllers, achieving the highest performance with minimal computational effort.
Nowadays, liquid rocket engines use closed-loop control at most near steady operating conditions. The control of the transient phases is traditionally performed in open-loop due to highly nonlinear system dynamics. This situation is unsatisfactory, in particular for reusable engines. The open-loop control system cannot provide optimal engine performance due to external disturbances or the degeneration of engine components over time. In this paper, we study a deep reinforcement learning approach for optimal control of a generic gas-generator engine's continuous start-up phase. It is shown that the learned policy can reach different steady-state operating points and convincingly adapt to changing system parameters. A quantitative comparison with carefully tuned open-loop sequences and PID controllers is included. The deep reinforcement learning controller achieves the highest performance and requires only minimal computational effort to calculate the control action, which is a big advantage over approaches that require online optimization, such as model predictive control. control.