Rocket Engine Control with Deep Reinforcement Learning

Kurzfassung

The space industry is transitioning to reusable and cost-efficient launch vehicles. This significant transformation creates new operational challenges for liquid propellant rocket engines. Reusable launch vehicles require precise and fast thrust control over a wide throttling range, while component aging with each flight can lead to changes in engine system dynamics due to wear and tear. In addition, the increasing use of low-cost additive manufacturing introduces variability in hardware geometry, creating additional uncertainties. The transition to more efficient but more complex engine cycles, such as staged combustion, and the use of clustered engine configurations further complicate operations. Closed-loop control systems offer a promising solution by providing accurate thrust control even as system dynamics change due to engine reuse and manufacturing variability. However, designing classical controllers for liquid propellant rocket engines is challenging because it requires the simultaneous control of multiple coupled variables, while the computational power of space-qualified processors is limited. As a result, there is a growing need for new advanced control algorithms suitable for reusable engines. One promising solution is to use neural network-based controllers trained with deep reinforcement learning on a simulation model of the engine: They require minimal computational power during deployment and can learn optimal control policies for complex nonlinear systems. This thesis therefore investigates the suitability of deep reinforcement learning for liquid rocket engine control and compares the results with state-of-the-art model predictive control. Two test cases around the LOX/LNG expander-bleed LUMEN engine are studied, each presenting different control challenges, including thrust control over a wide throttling range, constraint handling, and maximizing engine efficiency. A controller is also being experimentally tested at the P8.3 test facility in Lampoldshausen, Germany. The controller achieved promising accuracy in controlling multiple variables simultaneously with an average error of only 1.3 %, demonstrating the viability of the approach. This work also discusses stability and robustness considerations for future engine control systems, where safety and reliability are paramount.