Applying DLR´s Software Engineering Guidelines through an LLM for CI Pipeline and Software Testing

Abstract

This thesis explores the ability of Large Language Models, specifically GPT-4, to automate key tasks in software engineering: generating continuous integration pipelines and software tests. Using the open-source GitLab-Calendar project from the German Aerospace Center (DLR) as a case study, the generated outputs are evaluated for both functionality and adherence to the DLR Software Engineering Guideline. Detailed prompts were prepared to provide the necessary context, project structure, and guideline requirements to the LLM for generating accurate responses.

The results show that while GPT-4 can quickly generate CI pipelines and software tests with proper structure, the outputs were not fully functional and did not meet all guideline recommendations. The generated CI pipeline encountered errors due to wrong assumptions about project dependencies, and the software tests were based on a misunderstanding of the project’s code structure. However, GPT-4 demonstrated an understanding of industry best practices, and under human oversight, it has the potential to speed up the development process by automating boilerplate code and configuration files.

This study highlights both the strengths and limitations of LLMs in software engineering, showing the importance of iterative prompt refinement and human intervention to correct errors and optimize results. Although LLMs can provide templates and improve productivity, they are not yet capable of fully replacing human developers in complex software projects. The thesis concludes by discussing the results of the study and suggesting areas for further research.