Hernandez-Hernandez, Hugo (2023) Masked Image Modeling for Representation Learning in Earth Observation. Masterarbeit, TU Munich.
![]() |
PDF
7MB |
Kurzfassung
Deep learning applied to Earth observation (EO) yields impressive results. However, a significant challenge in EO is the rapidly increasing data volume while limited annotation resources available. Self-supervised representation learning (SSL) employs large amounts of unlabeled data. Recently, Masked Image Modelling (MIM) demonstrated scalability in model and data size. MIM masks a defined ratio of the input image for training a model to predict the masked patches. The learnt encoder is transferred to downstream tasks. In this work, we explore a new approach of MIM for EO combining two state-of-the-art SSL methodologies. One employs the Masked Autoencoder (MAE), which asymmetrically masks and reconstructs the raw input with the aid of an encoder operating on the visible patches followed by a smaller decoder reconstructing. The second methodology utilizes the Masked Feature Prediction (MFP), where image feature descriptors get reconstructed. We test our approach on the (SSL4E0-s12) dataset reconstructing Histogram Oriented Gradients (HOG). We evaluate the pre-trained model on a multi-class classification for Eurosat. Experimental results indicate stable performance with more than 90% accuracy down to 10% of labeled data. An ablation study on data normalization reveals that linear classification downstream task accuracy benefits from normalization by up to 6%. In contrast, fine tuning accuracies are robust to data normalization.
elib-URL des Eintrags: | https://elib.dlr.de/194890/ | ||||||||
---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Hochschulschrift (Masterarbeit) | ||||||||
Titel: | Masked Image Modeling for Representation Learning in Earth Observation | ||||||||
Autoren: |
| ||||||||
Datum: | 2023 | ||||||||
Referierte Publikation: | Nein | ||||||||
Open Access: | Ja | ||||||||
Seitenanzahl: | 73 | ||||||||
Status: | veröffentlicht | ||||||||
Stichwörter: | self-supervised learning, Earth observation, image compression, artificial intelligence, deep learning, EuroSAT | ||||||||
Institution: | TU Munich | ||||||||
Abteilung: | TUM School of Engineering and Design | ||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||
HGF - Programm: | Raumfahrt | ||||||||
HGF - Programmthema: | Erdbeobachtung | ||||||||
DLR - Schwerpunkt: | Raumfahrt | ||||||||
DLR - Forschungsgebiet: | R EO - Erdbeobachtung | ||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | R - Künstliche Intelligenz | ||||||||
Standort: | Oberpfaffenhofen | ||||||||
Institute & Einrichtungen: | Institut für Methodik der Fernerkundung > EO Data Science | ||||||||
Hinterlegt von: | Albrecht, Conrad M | ||||||||
Hinterlegt am: | 28 Apr 2023 10:56 | ||||||||
Letzte Änderung: | 07 Jul 2023 11:03 |
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags