From ConOps to an Operational Domain Model: Harnessing LLMs for Conceptual Model Design

Abstract

Safety-critical domains require a rigorous assessment before deployment. AI further exacerbates the issue. Several organizations are currently working on recommended practices to ensure the development and safety of AI-based systems. These practices often recommend identifying the high-level functioning of the system first, e.g., in the form of a Concept of Operations (ConOps), and then deriving certain artifacts. These artifacts, such as the Operational Design Domain and scenarios are used to specify data sets. In the computational space, the Operational Domain Model (ODM) can be represented using XML. The ODM is a single point of knowledge from which all subsequent specifications are generated, and, thus, must be designed carefully by a domain expert. For complex systems, the process can be complicated, manually intensive, and prone to errors. An automation of this step would greatly aid to achieve consistency and correctness of the ODM. This paper leverages the use of Large Language Models (LLMs) to transform the ConOps defined in natural language into an ODM representation in the form of XML. The later will capture all the critical elements of an ODM and will also be validated using the LLMs process, thereby helping to achieve traceability with the operational requirements in ConOps. Automated ODM generation significantly reduces the effort required to generate datasets for AI systems, ensuring consistency and correctness. This would also amplify the potential for automating the entire process, from the ConOps to systematic data generation. Properties like traceability will significantly enhance the provision of safety guarantees for AI systems.