On-line diagnostic tools for high resolution modelling

This is the latest version of this item.

Abstract

Input and output (I/O) of simulation data is an important aspect in high performance computing (HPC). Especially in high resolution numerical simulations over large domains with a high degree of distributed memory parallelisation I/O becomes a limiting factor. In the last years computational throughput of HPC systems increased, opening the ability to operate numerical weather and climate models with increasingly finer resolutions. This results in increasing amounts of data output from the model system. From the user's perspective the problems arise, where to store and archive these data, and which tools to use for subsequent post-processing. Transferring large data between different systems and subsequent processing require additional I/O operations, repeatedly hitting the I/O bottleneck and also memory limits of the computing system, if postprocessing is performed serially. One way to deal with I/O limitations is reducing the amount of output data by coarse-graining, although this may abandon advantages previously gained through the finer resolved simulations in subsequent processing. In contrast, on-line diagnostic tools use the fully resolved data, perform data reduction, and enable advanced diagnostics, which are not applicable off-line. We introduce a general and standardised interface to include sophisticated on-line diagnostic tools into the ICOsahedral Non-hydrostatic (ICON) modelling framework based on the Modular Earth Submodel System (MESSy). We present results of a novel complex on-line diagnostic tool calculating joint-PDFs on a coarse regular grid from high-resolution simulations. This also includes runtime analyses to evaluate additional computational costs from the on-line diagnostic and from the diagnostic interface itself. Furthermore, we give an overview of diagnostic tools, which were implemented or are currently under development using the diagnostic interface. We also present an outlook of future projects with enhanced on-line diagnostics. The presented work helps to overcome some of the limitations, which occur when going to even higher resolutions. The standardised interface and the on-line diagnostics will provide helpful and sophisticated tools for process oriented analyses of high resolution simulations, and will help to reduce the user's burden of handling large data from high resolution simulations.

Available Versions of this Item

• On-line diagnostic tools for high resolution modelling. (deposited 29 Mar 2016 11:10) [Currently Displayed]