elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Early and late fusion of multiple modalities in Sentinel imagery and social media retrieval

Yao, Wei and Moumtzidou, Anastasia and Dumitru, Corneliu Octavian and Stelios, Andreadis and Gialampoukidis, Ilias and Vrochidis, Stefanos and Datcu, Mihai and Kompatsiaris, Ioannis (2021) Early and late fusion of multiple modalities in Sentinel imagery and social media retrieval. ICPR 2020, 10.-15.01.2021, online.

[img] PDF
6MB

Abstract

Discovering potential concepts and events by analyzing Earth Observation (EO) data may be supported by fusing other distributed data sources such as non-EO data, for instance, in-situ citizen observations from social media. The retrieval of relevant information based on a target query or event is critical for operational purposes, for example, to monitor flood events in urban areas, and crop monitoring for food security scenarios. To that end, we propose an early-fusion (low-level features) and late-fusion (high-level concepts) mechanism that combines the results of two EU-funded projects for information retrieval in Sentinel imagery and social media data sources. In the early fusion part, the model is based on active learning that effectively merges Sentinel-1 and Sentinel-2 bands, and assists users to extract patterns. On the other hand, the late fusion mechanism exploits the context of other georeferenced data such as social media retrieval, to further enrich the list of retrieved Sentinel image patches. Quantitative and qualitative results show the effectiveness of our proposed approach.

Item URL in elib:https://elib.dlr.de/138091/
Document Type:Conference or Workshop Item (Poster)
Title:Early and late fusion of multiple modalities in Sentinel imagery and social media retrieval
Authors:
AuthorsInstitution or Email of AuthorsAuthor's ORCID iD
Yao, WeiWei.Yao (at) dlr.deUNSPECIFIED
Moumtzidou, Anastasiamoumtzid (at) iti.grUNSPECIFIED
Dumitru, Corneliu OctavianCorneliu.Dumitru (at) dlr.deUNSPECIFIED
Stelios, Andreadisandreadisst (at) iti.grUNSPECIFIED
Gialampoukidis, Iliasheliasgj (at) iti.grUNSPECIFIED
Vrochidis, Stefanosstefanos (at) iti.grUNSPECIFIED
Datcu, MihaiMihai.Datcu (at) dlr.deUNSPECIFIED
Kompatsiaris, Ioannisikom (at) iti.grUNSPECIFIED
Date:January 2021
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In SCOPUS:No
In ISI Web of Science:No
Status:Accepted
Keywords:Multimodal data fusion, Sentinel imagery retrieval, Social media retrieval, Earth Observation, Big Data
Event Title:ICPR 2020
Event Location:online
Event Type:international Conference
Event Dates:10.-15.01.2021
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Earth Observation
DLR - Research theme (Project):R - Optical remote sensing
Location: Oberpfaffenhofen
Institutes and Institutions:Remote Sensing Technology Institute > EO Data Science
Deposited By: Yao, Wei
Deposited On:26 Nov 2020 16:03
Last Modified:26 Nov 2020 16:03

Repository Staff Only: item control page

Browse
Search
Help & Contact
Information
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.