Scientific Software Integration and Data Management with TENT and DataFinder

Kurzfassung

The numerical simulation of real-world problems becomes increasingly complex. On the computational side there is a trend towards coupled simulation of several physical phenomena (e.g., fluid-structure interaction). Additional computational components, such as pre-processing, computational steering, post-processing, and visualization add to the complexity of the resulting workflow. If the workflow is to be executed in a distributed computing environment or requires batch processing for certain components, direct control by the application engineer is a difficult and time consuming task. Integration systems provide a comfortable execution environment for this kind of application. Via a GUI on a desktop computer, the engineer graphically constructs the workflow, sets input parameters for individual components, and controls the workflow execution. As an example, the integration system TENT which was developed at the German Aerospace Center (DLR) is presented. Additionally to the features listed above, it allows online monitoring and steering of individual workflow components and co-operation of users at different sites. The management of experimental and simulation data is a growing problem for research institutes worldwide. At DLR, a thorough investigation, including practical proof-of-concept exercises, was performed to identify the most cost-effective solution in a research environment. The winning concept was developed into the data management tool DataFinder. Based on a client-server architecture and open standards (WebDAV, XML etc.), it allows to create application-specific data types (e.g., "CFD simulation result"), and to store and retrieve the data together with associated meta-data. Additionally to data storage and retrieval, the DataFinder client supports relations (links) between data objects and provides complex search functions. TENT and DataFinder follow the same data management concept. If data produced by a simulation are stored in TENT, the simulation parameters are added as meta-data automatically. In DataFinder, the collection of data files corresponding to a TENT simulation can be handled as a single object, and search functions can be used to identify it by its associated simulation parameters.