Exploring the atmosphere composition with innovative online data analysis services integrating novel level-3 products from Copernicus Sentinel mission

Abstract

For more than 10 years, the EOC Geoservice (https://geoservice.dlr.de) is operational and provides access to all its hosted data collections and products via the OGC-compliant interfaces WMS and WCS. ISO metadata on data collections and products are exposed via compliant catalogue services. The EOC Geoservice operationally provides access to a multitude of operational EO and EO related produtcts. Among these are specific Sentinel-2-products, value-added land-surface products such as the World Settlement Footprint and a variety of atmosphere-relateted products from GOME-2/MetOp-A/B/C Level-3 as well as innovative Level 3 trace gas, cloud and radiation products derived from Sentinel-5P/TROPOMI observations. The EOC Geoservice faces the problem that it publishes its extremely heterogenious data collections and product though a number of different catalogue services. The challenge for EOC Geoservice is to provide harmonized and standardized metadata for its complete variety of products to support state-of-the-art data discovery. Moreover, state-of-the-art data access to support scientific data analysis by data cube approaches is required. The solution which is implemented by the EOC Geoservice extends the existing OGC Web Service APIs with an additional STAC (SpatioTemporal Asset Cataloguem https://stacspec.org) EO product catalog. This interface allows to easily access published datasets via data cube concepts, supporting direct integration in operational processing environments or into interactive Jupyter notebooks. The aforementioned additional STAC EO product catalog is based on the existing catalog entries in the database. By using templates, they are converted into one of the following five standards: OpenSearch XML EO (to support existing applicatons), OpenSearch JSON, STAC (JSON) and STAC (HMTL). This conversion is performed on-the-fly, all required information for collection and product metadata is stored in an internal Postgres-database. In order to facilitate the uptake of new products (such as individual L3 atmosphere products from S5P/TROPOMI) by the EOC Geoservice, configurable tools for the automated extraction of relevant metadata information from CF-compliant NetCDF files are currently in development. As other software components, this tools is going to be published as open source. For all technologies, the GeoServer software is the technical backbone. Throughout the above mentioned 10-year period, DLR has significantly supported its further-development as open-source software. Its most recent improvements (integration of STAC) have been funded in the framework of the ESA GSTP-project Technologies for the Management of Long EO Data Time Series (LOOSE). Integration of all innovative interfaces into an operational data discovery, access and analysis service (EOC Geoservice and DataCube) for the Copernicus atmospheric composition missions Sentinel-5P, Sentinel-4 and Sentinel-5 is supported by the DLR programmatic project Innovative Produktentwicklung zur Analyse der Atmosphärenzusammensetzung (INPULS).