Workflow Orchestration for Data Analysis Processes in Aerospace - Streamlining Incremental Flight Analysis during the Callisto Campaign

Abstract

As part of the incremental flight analysis during the CALLISTO campaign, efficient workflows are essential for data analysis in order to evaluate large amount of data in a fast and reproducible manner. The goal of this thesis is the development and evaluation of a prototype for a flexible orchestration environment, that would not only allow for an easy development of workflows, but also the integration of already existing programs, scripts and models into an automated process. Based on Prefect a modular infrastructure was introduced, that allows for the demonstration of exemplary, but founded workflows while enabling expandability for future use cases. This thesis includes the requirement analysis, the implementation and setup of the orchestration system and the presentation of the two exemplary showcases. The results of these showcases show that the prototype improves both transparency and efficiency without the need of discarding existing practices and processes. The presented system provides a robust foundation for the long term optimization of workflows.