Exascale-ready adaptive mesh refinement and applications in Earth system modelling

Abstract

Increasing the resolution of the computational mesh is one of the most effective tools to boost the accuracy of numerical earth system simulations and one of the current core challenges is enabling global sub-km scale simulations. However, increased mesh resolution comes at the cost of increased computational effort and memory consumption. With adaptive mesh refinement (AMR) the resolution can be dynamically controlled to use a fine resolution only when numerically necessary and keep a coarser mesh outside of regions of interest, hence reducing the required resources by orders of magnitude. Especially when the mesh refinement changes dynamically in time, efficiently managing the mesh and simulation data in parallel becomes a major challenge on its own. Modern space-filling curve (SFC) techniques are well-suited for this task due their low-memory footprint and fast scalable management algorithms. Previously, these techniques were only available for hexahedral or 2D triangular element shapes. We demonstrate the t8code library for massively scalable AMR that extends SFCs to all classic element shapes (Quadrilaterals, Triangles, Hexahedra, Tetrahedra, Prisms, Pyramids) and hence offers efficient and scalable mesh management for a wide variety of simulations. Our algorithms scale to over 1e12 mesh elements and 1 million parallel processes. In the ongoing Helmholtz incubator project Pilot Lab Exascale Earth System Modelling (PL-ExaESM) we couple t8code with the Modular Earth Submodel System framework MESSy in order to reduce the output file size of atmospheric chemistry simulations. In this talk we present our first results and discuss further plans with AMR in ESM.