Machine Learning-based Reentry Guidance for the ReFEx Mission: Analysis and Comparison of Reinforcement Learning and Genetic Programming

Abstract

This research applies two Machine Learning (ML) techniques, Deep Reinforcement Learning (DRL) and Genetic Programming (GP), to the task of generating the online guidance command for the REusability Flight EXperiment (ReFEx) vehicle. The focus is on the atmospheric flight phase of the reentry mission, in which the models trained via DRL and GP are used to produce real-time corrections for the reference guidance command computed beforehand. This further update stage is necessary to account for external disturbances and modeling errors. The ML models are tested and validated in the 6 Degrees Of Freedom (DOF) high fidelity simulator developed for the verification and validation of the Guidance Navigation and Control (GNC) subsystem of the ReFEx mission. Both methods are compared to each other and to the baseline optimization-based guidance algorithm to assess the performances and applicability of ML techniques to a real mission. DRL and GP are chosen for their complementary features: DRL has been used to train a Multilayer Perceptron (MLP) which results in a black-box model, whereas GP can produce a human-readable continuous and differentiable model, which is available as a symbolic expression. Results show that the DRL can deliver the same performances as the baseline guidance algorithm, while the GP achieves slightly worse results still comparable to it. Moreover, both techniques feature online execution times approximately three orders of magnitude faster than the baseline optimization-based strategy.