Validation of the KP Labs Multi-Class Cloud Segmentation Processor and Its Comparison to the Sentinel‑2 L2A Scene Classification Map

Kurzfassung

New classifiers developed using machine learning techniques, designed for global monitoring applications face the challenge of generalizing across a wide variety of biomes and across seasonal changes. Satellite images offer large, potentially global spatial scalability, but they might be contaminated by clouds – such cloudy images can be removed from further downstream processing and downlinking. Therefore, building accurate cloud segmentation algorithms are of practical importance, and may be used as smart on-board/on-the-ground data selection/compression steps. In this context, and as part of the Artificial Intelligence for Quality Control (AI4QC) activity supported by the European Space Agency (ESA), KP Labs recently developed a deep learning classifier built upon the Automated Machine Learning (AutoML) paradigms. Operating on all spectral bands of Sentinel‑2 imagery, the classifier produces multi‑class cloud‑classification products at 10‑meter spatial resolution, providing the following classes: clear‑sky, thick cloud, thin cloud, and cloud shadow.

Validation is performed using two reference datasets: fixed-sites, comprising 12 scenes from five fixed locations that represent five different biomes, and random-sites, comprising 13 randomly selected scenes from around the globe. Both datasets cover a wide variety of biomes captured across multiple seasons in 2024 by the Sentinel‑2 mission. These datasets were manually annotated by an expert through photointerpretation, comprehending ~677.000 reference pixels. They test the classifier’s performance under diverse land surface conditions (bare soil, vegetation, water, snow) and atmospheric conditions. Validation results are compared with the Scene Classification Layer (SCL) of the baseline Sen2Cor processor, which is distributed in the L2A products of the Sentinel 2 mission. Robust performance metrics for imbalanced classification, such as the Normalized Matthews Correlation Coefficient (nMCC), are used, ranging from 0 (total misclassification) to 1 (perfect prediction), with 0.5 indicating no better performance than random guessing. In addition, misclassification patterns are analyzed with confusion matrices, spider plots, and a qualitative assessment through visual inspection.

The results show an overall improvement over Sen2Cor of +1 percentage point (p.p.) (from 0.89 to 0.90 in nMCC) for the fixed sites dataset. In the other hand, a significant improvement of +5 p.p. (from 0.86 to 0.91 in nMCC) was observed for the random-sites dataset, with the greatest gain for the cloud shadow class (+9 p.p., from 0.87 to 0.96). The qualitative assessment indicates a general reduction in noise within the classification maps, likely due to the ability of the classifier to appropriately capture spatial-spectral features of the Sentinel-2 image data, and suggests specific improvements such as better cloud shadow detection over snow. These results translate to better filtering for downstream analysis applications and improved decision making.