Pixel-Level Classification of Spaceborne Multi- and Hyperspectral Imagery: A Deep Learning Approach

Abstract

This research aims to enhance the classification algorithms used in Atmospheric Correction (AC) by leveraging the spatial-spectral feature extraction capabilities of state-of-the-art Deep Learning (DL) models for pixel-level classification of spectral imagery. The primary objective is to advance the current state-of-the-art in preparation for the next generation of hyperspectral sensors. The initial phase of this work focuses on the integration of threshold-based masking algorithms, widely applied in multispectral remote sensing, with Convolutional Neural Networks (CNNs). This integration demonstrates significant improvements in classification outcomes, by combining traditional approaches with modern deep learning techniques.

As part of this research, guidelines are established for harmonized labeling criteria based on physical parameters, which are essential for creating robust training and testing datasets that incorporate multiple sources of information like manual labeling and products from the Tropospheric Monitoring Instrument (TROPOMI) sensor onboard the Sentinel-5P mission. This work uses the Python-based Atmospheric Correction (PACO) software developed by the German Aerospace Center (DLR) and conducts experiments using data from the multispectral Sentinel-2 mission and the Environmental Mapping and Analysis Program (EnMAP), which captures hyperspectral imagery.

For the multispectral case, the proposed models consistently outperform the PACO baseline across a variety of testing datasets, showing improvements of up to 18.3 percentage points in normalized Matthew's Correlation Coefficient (nMCC). The largest gains were observed in challenging classification scenarios, particularly in the shadow and clear classes.

In the hyperspectral case, classification results were translated into physical parameters by comparing the predicted cloud fractions with those from TROPOMI. While global regression metrics showed comparable performance between the CNN-based methods and the PACO baseline, the main contribution lies in the analysis of the strengths and weaknesses of the proposed validation methodology. In particular, the use of Accuracy, Precision and Uncertainty (APU) plots enabled a deep assessment of model behavior across different cloud fraction ranges, highlighting limitations in the reference data and the effects of spatial and temporal mismatches.

By addressing the challenges in dataset preparation and using cutting-edge neural network models, this work provides insights for obtaining improved classification products for multi- and hyper-spectral remote sensing missions with a global implementation perspective.