Knowledge Graph-Enhanced Retrieval-Augmented Generation for Earth Observation Data (Publication)

Abstract

Large language models have strong capabilities for different purposes, such as searching and question-answering. However, they hallucinate on domain-specific tasks, leading to potential risks such as misinformation spread or decay of trust between the technology and the user. These risks are more or less prominent depending on the context where an LLM is employed. For example, in a scientific setting, such as the Earth Observation (EO) domain, the LLM must ensure important criteria when answering, such as depth and groundedness [2]. To overcome hallucinations, recent research indicates that compound systems, which employ external tools and knowledge alongside LLMs, outperform standalone LLMs. Therefore, this paper presents a compound system to create a question-answering model for the EO domain. More precisely, our approach employs a Retrieval-Augmented Generation (RAG)-based model, focusing on three sequential components: (1) Data Curation to enable the LLM to access a semantically interconnected multi-genre corpora (e.g., scientific and datasets) when answering a question. (2) RAG-Based Model to balance between the LLM's existing knowledge and the curated data (from (1)). Lastly, (3) LLM-Based Evaluation to compare standalone LLM answers to our RAG-based model. Our evaluation across 70 EO questions shows that our approach achieves the highest score across all criteria (e.g., helpfulness), whereas traditional RAG underperforms zero-shot prompting on larger models. Our code and data are available on GitHub (https://github.com/DLR-SC/RAG-for-Earth-Observation) and Zenodo (https://doi.org/10.5281/zenodo.17106948).