elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Impressum | Datenschutz | Kontakt | English
Schriftgröße: [-] Text [+]

Can linguistic features extracted from geo-referenced tweets help building function classification in remote sensing?

Häberle, Matthias und Hoffmann, Eike Jens und Zhu, Xiao Xiang (2022) Can linguistic features extracted from geo-referenced tweets help building function classification in remote sensing? ISPRS Journal of Photogrammetry and Remote Sensing, 188, Seiten 255-268. Elsevier. doi: 10.1016/j.isprsjprs.2022.04.006. ISSN 0924-2716.

[img] PDF - Verlagsversion (veröffentlichte Fassung)
5MB

Offizielle URL: https://www.sciencedirect.com/science/article/pii/S0924271622001058

Kurzfassung

The fusion of two or more different data sources is a widely accepted technique in remote sensing while becoming increasingly important due to the availability of big Earth Observation satellite data. As a complementary source of geo-information to satellite data, massive text messages from social media form a temporally quasi-seamless, spatially multi-perspective stream, but with unknown and diverse quality. Despite the uncontrolled quality: can linguistic features extracted from geo-referenced tweets support remote sensing tasks? This work presents a straightforward decision fusion framework for very high-resolution remote sensing images and Twitter text messages. We apply our proposed fusion framework to a land-use classification task - the building function classification task - in which we classify building functions like commercial or residential based on linguistic features derived from tweets and remote sensing images. Using building tags from OpenStreetMap (OSM), we labeled tweets and very high-resolution (VHR) images from Google Maps. We collected English tweets from San Francisco, New York City, Los Angeles, and Washington D.C. and trained a stacked bi-directional LSTM neural network with these tweets. For the aerial images, we predicted building functions with state-of-the-art Convolutional Neural Network (CNN) architectures fine-tuned from ImageNet on the given task. After predicting each modality separately, we combined the prediction probabilities of both models building-wise at a decision level. We show that the proposed fusion framework can improve the classification results of the building type classification task. To the best of our knowledge, we are the first to use semantic contents of Twitter messages and fusing them with remote sensing images to classify building functions at a single building level.

elib-URL des Eintrags:https://elib.dlr.de/187155/
Dokumentart:Zeitschriftenbeitrag
Titel:Can linguistic features extracted from geo-referenced tweets help building function classification in remote sensing?
Autoren:
AutorenInstitution oder E-Mail-AdresseAutoren-ORCID-iDORCID Put Code
Häberle, MatthiasMatthias.Haeberle (at) dlr.deNICHT SPEZIFIZIERTNICHT SPEZIFIZIERT
Hoffmann, Eike JensEikeJens.Hoffmann (at) dlr.deNICHT SPEZIFIZIERTNICHT SPEZIFIZIERT
Zhu, Xiao Xiangxiao.zhu (at) dlr.deNICHT SPEZIFIZIERTNICHT SPEZIFIZIERT
Datum:Juni 2022
Erschienen in:ISPRS Journal of Photogrammetry and Remote Sensing
Referierte Publikation:Ja
Open Access:Ja
Gold Open Access:Nein
In SCOPUS:Ja
In ISI Web of Science:Ja
Band:188
DOI:10.1016/j.isprsjprs.2022.04.006
Seitenbereich:Seiten 255-268
Verlag:Elsevier
ISSN:0924-2716
Status:veröffentlicht
Stichwörter:Remote sensing Decision fusion Building function classification Deep learning Natural language processing Word embedding
HGF - Forschungsbereich:Luftfahrt, Raumfahrt und Verkehr
HGF - Programm:Raumfahrt
HGF - Programmthema:Erdbeobachtung
DLR - Schwerpunkt:Raumfahrt
DLR - Forschungsgebiet:R EO - Erdbeobachtung
DLR - Teilgebiet (Projekt, Vorhaben):R - Künstliche Intelligenz
Standort: Oberpfaffenhofen
Institute & Einrichtungen:Institut für Methodik der Fernerkundung > EO Data Science
Hinterlegt von: Häberle, Matthias
Hinterlegt am:06 Jul 2022 13:47
Letzte Änderung:20 Jul 2022 11:12

Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags

Blättern
Suchen
Hilfe & Kontakt
Informationen
electronic library verwendet EPrints 3.3.12
Gestaltung Webseite und Datenbank: Copyright © Deutsches Zentrum für Luft- und Raumfahrt (DLR). Alle Rechte vorbehalten.