Assurance Strategies for Safe AI Operation: Model-based Framework for the Development, Simulation and Monitoring of ODD

Kurzfassung

There is a significant demand for the application of artificial intelligence in the aviation sector, to handle the ongoing traffic growth and solve scalability issues and thus increasing complexity. However, any utilization of AI must be trustworthy, and its integration into aircraft must never compromise safety. Current aviation is built upon deterministic systems that follow programmed rules, i.e., ones that behave predictably and can be verified through testing and certification. The existing framework for developing and certifying software systems in the aviation domain does not align with the development of AI systems, which utilize machine learning (ML) technology and neural networks. One of the core principles of such software is that training data defines the behavior. The same software can behave differently for different training, which completely challenges the existing standards and development practices in the aviation domain. To address this, EASA has developed first guidance on the certification of ML for aviation systems [1], and is continuing to work on the integration and safety of ML systems. Figure 1: Overview of the model-based ODD framework. One prerequisite is that the ML function is used only when it is safe to use. For ML systems this means that the environmental conditions during operation match Those represented in the training data. These operating conditions are formally defined as the operational design domain (ODD) of the system. Figure 2: Overview of the IAM-OSA simulation setup, utilizing SenSim2 framework. This work is the ongoing research of the project IAM-OSA and discusses the use of model-based approaches to define, test and supervise the ODD. We combine work on modeling the ODD, utilizing constraints and dependencies, for generating operational scenarios [1] with a further application of monitoring and safety supervision of the operation [2, 3]. We develop a model-based tool used to model the domain, refine and extract a formal model of the ODD, see figure 1. This ODD is then extracted, conforming to the ASAM OpenODD® standard. The standard was developed for the automotive domain, but can be adapted for the aviation domain, following a defined schema for syntax and taxonomy of the ODD. With this ODD, two goals are targeted to achieve. The first goal is to generate scenarios within and on the edges of the ODD, as well as evaluating the coverage of the scenarios. The second goal is to build a monitoring solution that can supervise the ODD and its properties, see figure 2. These goals are combined with the simulation framework SenSim2, which our Institute develops. The simulation framework is capable of simulating diverse ODD properties using the Unreal Engine 5, see figure 3. Using this strategy, we are developing a framework for the model-based development, extraction, simulation, and monitoring of ODD. With this framework, we aim to gain new insights into the concept of ODD, and its role in AI development and certification guidelines. The scenario simulation and monitoring framework can be utilized to test the edges of the ODD and refine its parameters. Figure 3, a-f: Simulation of different ODD parameters utilizing the SenSim2 framework, showcasing effects of rain, snow, fog, and daylight.