Evaluation of a Reinforcement Learning-Based System for Mid-Term Conflict Resolution in the Upper Airspace

Kurzfassung

One of the core tasks of air tra!c controllers (ATCOs) is to ensure safe separation between aircraft, and automation tools can support this objective through functions such as flight plan generation, conflict detection, and conflict resolution. This thesis focuses on mid-term conflict resolution in upper airspace. In recent years, various approaches have been proposed for this purpose, including geometrical algorithms, rule-based methods, genetic algorithms (GA), supervised learning, and reinforcement learning (RL). Among these, RL has gained significant attention due to its ability to learn decision-making policies through interaction with the environment. RL and GA pursue the same objective, namely the identification of the most e!cient solution. Furthermore, their underlying methodologies are comparable, as the reward function in RL plays a role similar to the fitness function in GA. The objective of this thesis is therefore to compare RL and GA in order to determine whether RL can provide more e!cient and higher-performing solutions. To this end, an RL-based system is designed and implemented within the VERA (Validation Environment for Researching ATC Tools) framework, a research-oriented simulation and validation platform for air tra!c control applications. The system is evaluated using VEHSi (Validation Environment based on Hybrid Simulations in Air Tra!c Control). At the end of the development process, the proposed system is compared with TraGAT, a solution based on GA. The comparison considers two main aspects: performance, measured in terms of computation time and resource consumption, and e!ciency, evaluated through trajectory length, fuel consumption, and CO2 emissions. Five conflict scenarios were used for the comparison, and both RLCoRe and TraGAT successfully resolved all conflicts. RLCoRe achieved better computational performance, requiring approximately 10 milliseconds and 495 MB of RAM, compared with 200 milliseconds and 850 MB for the default TraGAT configuration. The default TraGAT configuration achieved the highest e!ciency, with average fuel savings of approximately 71 kg compared with 27 kg for RLCoRe, although its solutions often deviated considerably from the initial flight plans. A second TraGAT configuration using the RouteSimilarity and RouteStructure optimization parameters preserved the original routes more closely, but required approximately 350 milliseconds and achieved only a small average fuel reduction of 1.21 kg, together with a slight average increase of 0.62 NM in trajectory length. Overall, RLCoRe o"ers a promising compromise between computational performance, e!ciency, command simplicity, and preservation of the initial flight plan