Data Quality as a Development Aspect of Save AI

Kurzfassung

As the use of AI continues to grow, more and more safety-critical systems are making use of AI technology. Ensuring the safety of AI systems is therefore an important task that also requires engineering methods. One of the key pillars of AI is the data used to set up the AI systems. Therefore, the quality of the data used to feed the AI model has a decisive impact on the quality of the AI system and thus on its safety. Degradation of the data quality (DQ) might have a high impact safeguarding critical systems. As an example, let us consider the training data for an AI system for automated driving does not contain a road sign or only contains it in poor image quality or the image is incorrectly labeled, the recognition of road signs will not work properly and thus endanger traffic. Therefore, well established concepts of DQ management are needed. This work takes these well-established concepts and set up a roadmap, which existing techniques should be considered for the different development stages defined by the Technical Readiness Level (TRL) system during the development of safety critical technology involving AI. In particular, this includes refining the requirements and the DQ model for the various TRLs, on that way also allowing to map relevant DQ characteristics to AI quality characteristic relevant for safety. Specifically, the data for a laboratory prototype (TRL 4) has different requirements than those for a field prototype (TRL 6-7) or those for a fully operational system (TRL 9). Therefore, the DQ model, which is composed of suitable DQ characteristics and DQ measures as defined, for example, in the upcoming ISO 5259 standard, can be kept simpler and thus examined in detail for lower TRLs and refined on the basis of the results obtained for higher ones. This refinement also applies to the data selection and optimization process. In certain development scenarios, synthetic data can be used for lower TRLs, which initially enables better and controllable data quality so that the requirements for the real data can be defined later. This work emphasizes the awareness of the DQ aspects in AI technique development and of the change of requirements during the different stages of the development process which allow to incorporate DQ in an end to end research and engineering implementation. All in all, this contributes to optimal DQ standards for the final system, which in turn help to ensure the different aspects of Safe AI in the development process.