Semantic enrichment of AI-detected buildings using remote sensing and multi-source geospatial data

Kurzfassung

Knowledge about the functional use and exact location of buildings is essential for a wide range of applications, including urban planning, natural hazard risk assessment, and mobility analysis. However, comprehensive, up-to-date, and large-scale building datasets that contain both precise geometry and information on building use remain scarce, even in data-rich countries like Germany. Existing data sources in Germany, such as the official cadastral data (LoD1 and LoD2), provide semantic information on building types, but they are often incomplete due to data gaps, inconsistencies across federal states, and outdated information. Similar problems exist in building data from OpenStreetMap (OSM): while offering crowd-sourced building types, both the data quality and completeness vary greatly by region. Consequently, parts of the built environment remain unmapped and lack functional labels, impeding data-driven urban analysis. To address these limitations, this study presents a scalable workflow that combines AI-based remote sensing techniques with multi-source data fusion to infer building use types across an entire metropolitan region. High-resolution aerial imagery and digital surface models are used to detect building footprints using a deep learning model based on a fully convolutional encoder-decoder architecture. Aerial imagery is used for the building detection because of its large-scale and up to date coverage, and its high spatial resolution. The model is trained and evaluated on annotated data from North Rhine-Westphalia and Berlin and achieves a mean intersection-over-union of over 92% on validation data. By experimenting with different training data sampling strategies, the model generalization is improved, and the best-performing configuration is applied to the full area of Berlin to extract accurate and up-to-date building geometries. This approach enables a systematic quantification of missing buildings in both cadastral and OSM building datasets. Since most building usage types are not discernable only based on aerial images, additional data are necessary to assign functional attributes to the detected buildings. These include both previously mentioned official cadastral data and multiple OpenStreetMap layers, such as buildings and land use polygons. The heterogeneous usage categories from both sources are harmonized and mapped onto four generalized functional classes: "residential", "commercial", "work-related" and "other". These categories reflect the primary daily travel purposes observed in urban areas, making them well-suited for urban mobility modeling. Adjacent buildings are spatially aggregated into building complexes and proportional type shares are assigned based on areal relationships from the external building datasets. For building complexes with no matching semantic data, functional approximations are derived from land use information. The resulting dataset provides proportional use shares across the four target categories for all buildings in the federal state of Berlin. This enables a nuanced representation of mixed-use as well as single-use structures. Beyond quantifying data gaps in existing building datasets, the study demonstrates how combining Earth observation data with volunteered and official sources can fill critical missing information on building location and use type. This approach offers a scalable solution for generating semantically enriched complete building inventories that reflect current urban conditions by bridging the gap between building geometry and semantic usage information.