DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Using social media images for building function classification

Hoffmann, Eike Jens and Abdulahhad, Karam and Zhu, Xiao Xiang (2023) Using social media images for building function classification. Cities, 133, p. 104107. Elsevier. doi: 10.1016/j.cities.2022.104107. ISSN 0264-2751.

[img] PDF - Published version

Official URL: https://www.sciencedirect.com/science/article/pii/S0264275122005467


Urban land use on a building instance level is crucial geo-information for many applications yet challenging to obtain. Steet-level images are highly suited to predict building functions as the building façades provide clear hints. Social media image platforms contain billions of images, including but not limited to street perspectives. This study proposes a filtering pipeline to yield high-quality, ground-level imagery from large-scale social media image datasets to cope with this issue. The pipeline ensures all resulting images have complete and valid geotags with a compass direction to relate image content and spatial objects. We analyze our method on a culturally diverse social media dataset from Flickr with more than 28 million images from 42 cities worldwide. The obtained dataset is then evaluated in the context of a building function classification task with three classes: Commercial, residential, and other. Fine-tuned state-of-the-art architectures yield F1 scores of up to 0.51 on the filtered images. Our analysis shows that the quality of the labels from OpenStreetMap limits the performance. Human-validated labels increase the F1 score by 0.2. Therefore, we consider these labels weak and publish the resulting images from our pipeline and the depicted buildings as a weakly labeled dataset

Item URL in elib:https://elib.dlr.de/194799/
Document Type:Article
Title:Using social media images for building function classification
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Hoffmann, Eike JensTU Münchenhttps://orcid.org/0000-0001-7702-0403UNSPECIFIED
Abdulahhad, KaramUNSPECIFIEDhttps://orcid.org/0000-0002-0041-7047UNSPECIFIED
Zhu, Xiao XiangUNSPECIFIEDhttps://orcid.org/0000-0001-5530-3613UNSPECIFIED
Date:February 2023
Journal or Publication Title:Cities
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:Yes
Page Range:p. 104107
Keywords:Social media image analysis; Big data analytics; Building function classification; Urban land use
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Earth Observation
DLR - Research theme (Project):R - Optical remote sensing, R - Artificial Intelligence
Location: Oberpfaffenhofen
Institutes and Institutions:Remote Sensing Technology Institute > EO Data Science
Deposited By: Knickl, Sabine
Deposited On:24 Apr 2023 14:41
Last Modified:15 May 2023 11:49

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.