Ait Ali Braham, Nassim und Albrecht, Conrad M und Gomes, Carlos und Blumenstiel, Benedikt (2025) Self-supervised learning for multispectral and hyperspectral remote sensing. 2025 ESA Living Planet Symposium, 2025-06-23 - 2025-06-27, Vienna, Austria.
Dieses Archiv kann nicht den Volltext zur Verfügung stellen.
Offizielle URL: https://lps25.esa.int/programme/programme-session/?id=ED4266A9-53C0-4768-A18A-7AA5C2CF00D9&presentationId=139B0C8E-479D-42A4-B999-0394E9CEE411
Kurzfassung
Foundation models have triggered a paradigm shift in computer vision and remote sensing, demonstrating remarkable generalizability across tasks with minimal fine-tuning. While extensive research has focused on RGB, multispectral, and radar imagery, the potential of foundation models on hyperspectral imagery remains largely untapped. Moreover, most existing works are limited to one or two sensors, despite the availability of diverse satellites capturing imagery across varying spectral ranges and spatial resolutions. Integrating different data sources can improve the versatility of foundation models and enables exploiting the complementarity of different sensors. In this work, we develop a foundation model for multispectral and hyperspectral data, leveraging the rich spectral information unique to HSI. Specifically, we pair the EnMAP-based SpectralEarth dataset with Sentinel-2 and Landsat-8 imagery to create a diverse, multi-sensor dataset for pre-training. Additionally, we design a self-supervised learning method based on masked image modeling and apply it to a vision transformer model. The employed architecture consists of a few initial sensor-specific layers to account for differences in spatial and spectral characteristics, followed by a shared vision transformer backbone that processes all modalities in a unified latent space. The pre-trained model is evaluated on downstream tasks from each sensor, including land cover and crop type classification, to assess its generalizability. Our findings highlight the effectiveness of multi-sensor pre-training and demonstrate positive transfer across modalities. This study contributes to ongoing efforts in developing generic foundation models and provides insights into the training of such models.
elib-URL des Eintrags: | https://elib.dlr.de/214938/ | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Konferenzbeitrag (Poster) | ||||||||||||||||||||
Titel: | Self-supervised learning for multispectral and hyperspectral remote sensing | ||||||||||||||||||||
Autoren: |
| ||||||||||||||||||||
Datum: | 26 Juni 2025 | ||||||||||||||||||||
Referierte Publikation: | Nein | ||||||||||||||||||||
Open Access: | Nein | ||||||||||||||||||||
Gold Open Access: | Nein | ||||||||||||||||||||
In SCOPUS: | Nein | ||||||||||||||||||||
In ISI Web of Science: | Nein | ||||||||||||||||||||
Status: | veröffentlicht | ||||||||||||||||||||
Stichwörter: | self-supervised learning, hyperspectral, EnMAP, multi-spectral, multi-modal fusion, Sentinel-1/2, Landsat 8 | ||||||||||||||||||||
Veranstaltungstitel: | 2025 ESA Living Planet Symposium | ||||||||||||||||||||
Veranstaltungsort: | Vienna, Austria | ||||||||||||||||||||
Veranstaltungsart: | internationale Konferenz | ||||||||||||||||||||
Veranstaltungsbeginn: | 23 Juni 2025 | ||||||||||||||||||||
Veranstaltungsende: | 27 Juni 2025 | ||||||||||||||||||||
Veranstalter : | European Space Agency | ||||||||||||||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||||||||||||||
HGF - Programm: | Raumfahrt | ||||||||||||||||||||
HGF - Programmthema: | Erdbeobachtung | ||||||||||||||||||||
DLR - Schwerpunkt: | Raumfahrt | ||||||||||||||||||||
DLR - Forschungsgebiet: | R EO - Erdbeobachtung | ||||||||||||||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | R - Künstliche Intelligenz, R - Optische Fernerkundung | ||||||||||||||||||||
Standort: | Oberpfaffenhofen | ||||||||||||||||||||
Institute & Einrichtungen: | Institut für Methodik der Fernerkundung > EO Data Science | ||||||||||||||||||||
Hinterlegt von: | Albrecht, Conrad M | ||||||||||||||||||||
Hinterlegt am: | 17 Jul 2025 11:46 | ||||||||||||||||||||
Letzte Änderung: | 04 Aug 2025 19:00 |
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags