DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Multi-sensor cloud and cloud shadow segmentation with a convolutional neural network

Wieland, Marc and Yu, Li and Martinis, Sandro (2019) Multi-sensor cloud and cloud shadow segmentation with a convolutional neural network. Remote Sensing of Environment, 230, pp. 1-12. Elsevier. DOI: 10.1016/j.rse.2019.05.022 ISSN 0034-4257

[img] PDF - Registered users only until 1 September 2021 - Postprint version (accepted manuscript)

Official URL: https://www.sciencedirect.com/science/article/abs/pii/S0034425719302159


Cloud and cloud shadow segmentation is a crucial pre-processing step for any application that uses multi-spectral satellite images. In particular, disaster related applications (e.g., flood monitoring or rapid damage mapping), which are highly time- and data-critical, require methods that produce accurate cloud and cloud shadow masks in short time while being able to adapt to large variations in the target domain (induced by atmospheric conditions, different sensors, scene properties, etc.). In this study, we propose a data-driven approach to semantic segmentation of cloud and cloud shadow in single date images based on a modified U-Net convolutional neural network that aims to fulfil these requirements. We train the network on a global database of Landsat OLI images for the segmentation of five classes (shadow, cloud, water, land and snow/ice). We compare the results to state-of-the-art methods, proof the models generalization ability across multiple satellite sensors (Landsat TM, Landsat ETM+, Landsat OLI and Sentinel-2) and show the influence of different training strategies and spectral band combinations on the performance of the segmentation. Our method consistently outperforms Fmask and a traditional Random Forest classifier on a globally distributed multi-sensor test dataset in terms of accuracy, Cohens Kappa coefficient, Dice coefficient and inference speed. The results indicate that a reduced feature space composed solely of red, green, blue and near-infrared bands already produces good results for all tested sensors. If available, adding shortwave-infrared bands can increase the accuracy. Contrast and brightness augmentations of the training data further improve the segmentation performance. The best performing U-Net model achieves an accuracy of 0.89, Kappa of 0.82 and Dice coefficient of 0.85, while running the inference over 896 test image tiles with 44.8 seconds/megapixel (2.8 seconds/megapixel on GPU). The Random Forest classifier reaches an accuracy of 0.79, Kappa of 0.65 and Dice coefficient of 0.74 with 3.9 seconds/megapixel inference time (on CPU) on the same training and testing data. The rule-based Fmask method takes significantly longer (277.8 seconds/megapixel) and produces results with an accuracy of 0.75, Kappa of 0.60 and Dice coefficient of 0.72.

Item URL in elib:https://elib.dlr.de/127745/
Document Type:Article
Title:Multi-sensor cloud and cloud shadow segmentation with a convolutional neural network
AuthorsInstitution or Email of AuthorsAuthors ORCID iD
Wieland, Marcmarc.wieland (at) dlr.dehttps://orcid.org/0000-0002-1155-723X
Yu, Liyu.li (at) dlr.deUNSPECIFIED
Martinis, Sandrosandro.martinis (at) dlr.dehttps://orcid.org/0000-0002-6400-361X
Date:15 May 2019
Journal or Publication Title:Remote Sensing of Environment
Refereed publication:Yes
Open Access:No
Gold Open Access:No
In ISI Web of Science:Yes
DOI :10.1016/j.rse.2019.05.022
Page Range:pp. 1-12
Keywords:Cloud; Cloud shadow; Convolutional neural network; Landsat; Sentinel-2
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Erdbeobachtung
DLR - Research theme (Project):R - Remote sensing and geoscience
Location: Oberpfaffenhofen
Institutes and Institutions:German Remote Sensing Data Center > Geo Risks and Civil Security
Deposited By: Wieland, Marc
Deposited On:19 Jun 2019 12:41
Last Modified:10 Sep 2019 09:03

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.