FAST-EO: Multi-Modal Foundation Models for Scalable Earth Observation and Earth Sciences

Kurzfassung

Earth Observation (EO) has become indispensable for monitoring and understanding environmental processes at regional and global scales. Recent advances in Geospatial AI Foundation Models (GFM) hold promise for significantly enhancing EO capabilities, particularly in the face of complex challenges such as climate change, resource management, and disaster mitigation [1]. Nonetheless, these advances must overcome several key hurdles, including scalability, multi-modal data fusion, and adaptability across diverse spatiotemporal contexts. In response, the FAST-EO project pioneers self-supervised, multi-modal foundation models designed to promote zero-shot generalization and cross-domain adaptation from large-scale EO datasets. A central innovation within FAST-EO is 4M4EO, an extended variant of the “Massively Multimodal Masked Modeling” framework [2]. By unifying optical, synthetic aperture radar (SAR), hyperspectral, and textual data in a single representation space, 4M4EO supports robust cross-modal transfer and generative capabilities, thereby reducing the need for extensive labeled datasets. In addition, 4M4EO incorporates time-series modeling techniques [3], which enhance its ability to capture dynamic environmental conditions such as floods, droughts, wildfires, and other climate-driven phenomena. Moreover, FAST-EO centralizes all use-case data and code under TerraTorch [4], an open-source framework that streamlines experimentation and ensures reproducibility. TerraTorch allows developers and end-users to seamlessly combine data preprocessing, model training, and evaluation pipelines, fostering collaborative development and transparent research practices. Through this unified ecosystem, diverse EO stakeholders—including scientists, policy makers, and industry practitioners—can more easily adopt advanced AI solutions for a wide range of applications. To showcase the effectiveness of these methodologies, FAST-EO targets several high-impact use cases: • Disaster Resilience: Rapid identification of floods and wildfires to support emergency planning and resource allocation. • Greenhouse Gas Monitoring: Detection of methane leaks using hyperspectral imaging to guide climate action strategies. • Forest Biomass & Carbon Accounting: Multi-sensor integration for tracking deforestation, reforestation, and overall carbon flux. • Soil Property Estimation: Precision agriculture enabled by retrieving key soil parameters from hyperspectral observations. • Land Cover Change: Detailed mapping of urban expansion, agricultural transitions, and ecological shifts to inform sustainable development. • Mining Expansion Monitoring: Large-scale detection and assessment of mining activities to quantify environmental impacts. By combining advanced AI architectures, high-performance computing, and open-source collaboration, FAST-EO illustrates how foundation models can revolutionize EO and environmental sciences without being constrained by traditional data bottlenecks. In doing so, the project drives a new generation of analytics and decision support tools, catalyzing progress in climate monitoring, resource stewardship, and sustainable development. [1] Zappacosta, A., Kuzu, R. S., Dumitru, C. O., Molina, D. E., Brunschwiler, T., Kienzler, R., Jakubik, J., Blumenstiel, B., Cavallaro, G., Kesselheim, S., Sedona, R., Wijata, A., Tulczyjew, L., Marek, D., & Nalepa, J. (2024, May 7–10). Democratizing foundation models for Earth Observation applications. ESA-ECMWF ML4ESOP Workshop, ESA-ESRIN, Frascati, Italy. [2] Mizrahi, D., Bachmann, R., Kar, O., Yeo, T., Gao, M., Dehghan, A., & Zamir, A. (2024). 4m: Massively multimodal masked modeling. Advances in Neural Information Processing Systems, 36. [3] Jakubik, J., Roy, S., Phillips, C. E., Fraccaro, P., Godwin, D., Zadrozny, B., Szwarcman, D., Gomes, C., Nyirjesy, G., Edwards, B., Kimura, D., Simumba, N., Chu, L., Mukkavilli, S. K., Lambhate, D., Das, K., Bangalore, R., Oliveira, D., Muszynski, M., ... Ramachandran, R. (2023). Foundation models for generalist geospatial artificial intelligence. arXiv. https://arxiv.org/abs/2310.18660 [4] BM Research. (2025). TerraTorch: An open-source framework for large-scale geospatial AI. GitHub repository. https://github.com/IBM/terratorch