elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Masked Image Modelling for Representation Learning in Earth Observation

Hernandez-Hernandez, Hugo and Wang, Yi and Albrecht, Conrad M and Zhu, Xiao Xiang (2023) Masked Image Modelling for Representation Learning in Earth Observation. HelmholtzAI annual conference, 2023-06-12 - 2023-06-14, DESY Hamburg, Germany.

[img] PDF
4MB

Official URL: https://eventclass.it/haic2023/scientific/online-program/session?s=S-02a#e89

Abstract

Deep learning applied to Earth observation (EO) yields impressive results. However, a significant challenge in EO is the rapidly increasing data volume while limited annotation resources available. Self-supervised representation learning (SSL) employs large amounts of unlabeled data. Recently, Masked Image Modelling (MIM) demonstrated scalability in model and data size. MIM masks a defined ratio of the input image for training a model to predict the masked patches. The learnt encoder is transferred to downstream tasks. In this work, we explore a new approach of MIM for EO combining two state-of-the-art SSL methodologies. One employs the Masked Autoencoder (MAE), which asymmetrically masks and reconstructs the raw input with the aid of an encoder operating on the visible patches followed by a smaller decoder reconstructing. The second methodology utilizes the Masked Feature Prediction (MFP), where image feature descriptors get reconstructed. We test our approach on the SSL4E0-S12 dataset reconstructing Histogram Oriented Gradients (HOG). We evaluate the pre-trained model on a multi-class classification for EuroSAT. Experimental results indicate stable performance with more than 90% accuracy down to 10% of labeled data. An ablation study on data normalization reveals that linear classification downstream task accuracy benefits from normalization by up to 6%. In contrast, fine tuning accuracies are robust to data normalization.

Item URL in elib:https://elib.dlr.de/195496/
Document Type:Conference or Workshop Item (Speech)
Title:Masked Image Modelling for Representation Learning in Earth Observation
Authors:
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Hernandez-Hernandez, HugoUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Wang, YiUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Albrecht, Conrad MUNSPECIFIEDhttps://orcid.org/0009-0009-2422-7289UNSPECIFIED
Zhu, Xiao XiangUNSPECIFIEDhttps://orcid.org/0000-0001-5530-3613UNSPECIFIED
Date:2023
Refereed publication:No
Open Access:Yes
Gold Open Access:No
In SCOPUS:No
In ISI Web of Science:No
Status:Published
Keywords:self-supervised learning, masked image modelling, Sentinel-1, Sentinel-2, deep learning
Event Title:HelmholtzAI annual conference
Event Location:DESY Hamburg, Germany
Event Type:national Conference
Event Start Date:12 June 2023
Event End Date:14 June 2023
Organizer:Helmholtz Association
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Earth Observation
DLR - Research theme (Project):R - Artificial Intelligence
Location: Oberpfaffenhofen
Institutes and Institutions:Remote Sensing Technology Institute > EO Data Science
Deposited By: Albrecht, Conrad M
Deposited On:07 Jul 2023 08:21
Last Modified:24 Apr 2024 20:55

Repository Staff Only: item control page

Browse
Search
Help & Contact
Information
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.