DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Model Fusion for Building Type Classification from Aerial and Street View Images

Hoffmann, Eike Jens and Wang, Yuanyuan and Werner, Martin and Kang, Jian and Zhu, Xiao Xiang (2019) Model Fusion for Building Type Classification from Aerial and Street View Images. Remote Sensing, 11 (11), 1259/1-1259/20. Multidisciplinary Digital Publishing Institute (MDPI). DOI: 10.3390/rs11111259 ISSN 2072-4292

[img] PDF - Published version

Official URL: http://dx.doi.org/10.3390/rs11111259


This article addresses the question of mapping building functions jointly using both aerial and street view images via deep learning techniques. One of the central challenges here is determining a data fusion strategy that can cope with heterogeneous image modalities. We demonstrate that geometric combinations of the features of such two types of images, especially in an early stage of the convolutional layers, often lead to a destructive effect due to the spatial misalignment of the features. Therefore, we address this problem through a decision-level fusion of a diverse ensemble of models trained from each image type independently. In this way, the significant differences in appearance of aerial and street view images are taken into account. Compared to the common multi-stream end-to-end fusion approaches proposed in the literature, we are able to increase the precision scores from 68% to 76%. Another challenge is that sophisticated classification schemes needed for real applications are highly overlapping and not very well defined without sharp boundaries. As a consequence, classification using machine learning becomes significantly harder. In this work, we choose a highly compact classification scheme with four classes, commercial, residential, public, and industrial because such a classification has a very high value to urban geography being correlated with socio-demographic parameters such as population density and income

Item URL in elib:https://elib.dlr.de/128115/
Document Type:Article
Title:Model Fusion for Building Type Classification from Aerial and Street View Images
AuthorsInstitution or Email of AuthorsAuthors ORCID iD
Hoffmann, Eike JensTU Münchenhttps://orcid.org/0000-0001-7702-0403
Wang, YuanyuanTUMhttps://orcid.org/0000-0002-0586-9413
Date:June 2019
Journal or Publication Title:Remote Sensing
Refereed publication:Yes
Open Access:Yes
Gold Open Access:Yes
In ISI Web of Science:Yes
DOI :10.3390/rs11111259
Page Range:1259/1-1259/20
Publisher:Multidisciplinary Digital Publishing Institute (MDPI)
Keywords:street view image; aerial image; model fusion; building type classification; building function; CNN; urban land use; land cover
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Erdbeobachtung
DLR - Research theme (Project):R - Vorhaben Geowissenschaftl. Fernerkundungs- und GIS-Verfahren
Location: Oberpfaffenhofen
Institutes and Institutions:Remote Sensing Technology Institute > EO Data Science
Deposited By: Hoffmann, Eike Jens
Deposited On:28 Jun 2019 10:51
Last Modified:14 Dec 2019 04:27

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.