DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

SAR-to-Optical Image Translation Based on Conditional Generative Adversarial Networks - Optimization, Opportunities and Limits

Fuentes Reyes, Mario and Auer, Stefan and Merkle, Nina Marie and Henry, Corentin and Schmitt, Michael (2019) SAR-to-Optical Image Translation Based on Conditional Generative Adversarial Networks - Optimization, Opportunities and Limits. Remote Sensing, 11 (17), pp. 1-19. Multidisciplinary Digital Publishing Institute (MDPI). DOI: 10.3390/rs11172067 ISSN 2072-4292

[img] PDF - Published version

Official URL: https://www.mdpi.com/2072-4292/11/17/2067


Due to its all time capability, synthetic aperture radar (SAR) remote sensing plays an important role in Earth observation. The ability to interpret the data is limited, even for experts, as the human eye is not familiar to the impact of distance-dependent imaging, signal intensities detected in the radar spectrum as well as image characteristics related to speckle or steps of post-processing. This paper is concerned with machine learning for SAR-to-optical image-to-image translation in order to support the interpretation and analysis of original data. A conditional adversarial network is adopted and optimized in order to generate alternative SAR image representations based on the combination of SAR images (starting point) and optical images (reference) for training. Following this strategy, the focus is set on the value of empirical knowledge for initialization, the impact of results on follow-up applications, and the discussion of opportunities/drawbacks related to this application of deep learning. Case study results are shown for high resolution (SAR: TerraSAR-X, optical: ALOS PRISM) and low resolution (Sentinel-1 and -2) data. The properties of the alternative image representation are evaluated based on feedback from experts in SAR remote sensing and the impact on road extraction as an example for follow-up applications. The results provide the basis to explain fundamental limitations affecting the SAR-to-optical image translation idea but also indicate benefits from alternative SAR image representations.

Item URL in elib:https://elib.dlr.de/129009/
Document Type:Article
Title:SAR-to-Optical Image Translation Based on Conditional Generative Adversarial Networks - Optimization, Opportunities and Limits
AuthorsInstitution or Email of AuthorsAuthors ORCID iD
Fuentes Reyes, MarioMario.FuentesReyes (at) dlr.dehttps://orcid.org/0000-0002-6593-5152
Auer, StefanStefan.Auer (at) dlr.dehttps://orcid.org/0000-0001-9310-2337
Merkle, Nina MarieNina.Merkle (at) dlr.dehttps://orcid.org/0000-0003-4177-1066
Henry, CorentinCorentin.henry (at) dlr.dehttps://orcid.org/0000-0002-4330-3058
Schmitt, Michaelm.schmitt (at) tum.dehttps://orcid.org/0000-0002-0575-2362
Date:3 September 2019
Journal or Publication Title:Remote Sensing
Refereed publication:Yes
Open Access:Yes
Gold Open Access:Yes
In ISI Web of Science:Yes
DOI :10.3390/rs11172067
Page Range:pp. 1-19
Publisher:Multidisciplinary Digital Publishing Institute (MDPI)
Series Name:Advances in Remote Sensing Image Fusion
Keywords:Synthetic Aperture Radar (SAR); deep learning; interpretation; generative adversarial networks
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Erdbeobachtung
DLR - Research theme (Project):R - Vorhaben hochauflösende Fernerkundungsverfahren
Location: Oberpfaffenhofen
Institutes and Institutions:Remote Sensing Technology Institute > Photogrammetry and Image Analysis
Deposited By: Fuentes Reyes, Mario
Deposited On:05 Sep 2019 13:18
Last Modified:14 Dec 2019 04:27

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.