Leveraging Deep Learning for Enhanced Building Information Retrieval and Reconstruction from Remote Sensing Imagery

Kurzfassung

In large cities around the world, city centres are often densely built-up. Due to the dense development, it is difficult to distinguish the buildings on the basis of satellite and aerial images and to reconstruct them in 3D. However, this is essential for applications such as urban planning, disaster management, solar energy potential assessment, flow simulation, and many others. Therefore, the goal of this dissertation is to provide methods for retrieving building information and reconstructing buildings in 3D. For this purpose, three methods are introduced. The first method segments building sections on satellite and aerial images, as well as associated Digital Surface Model (DSM), by segmenting the separation lines between them using a fully convolutional neural network (FCN). In addition, the remaining pixels assigned to the building are segmented. The building and separation line segments are then used to obtain seamless building sections using the watershed transformation. In addition, morphology is used to close existing gaps in separation lines. To further improve the separation line and building segments, a loss function is used, which results in the FCN already producing fewer gaps before morphology is applied and building segments have straighter edges and sharper corners. The resulting building sections are then further processed into polygons and level of detail (LoD)-1 models. The method is robust and works on both aerial and satellite data. In addition, it can segment building sections in complex scenarios more accurately than the compared approach. The method manages to accurately segment even in highly complex scenarios with very small building sections and informal settlement after the FCN has been re-trained with data from different cities with dense buildings. This is particularly important for crisis management and humanitarian aid. Moreover, a method is introduced which vectorizes building footprints and regularizes their outlines in two steps. In the first step, a deep neural network predicts the primary orientation angle of the building polygon. In the second step, all vertices are adjusted, such that their inside angles are either 90° or 180° with respect to the primary orientation. The second method, PLANES4LOD2, generates LoD-2 models based on aerial images and DSM, as well as digital terrain model (DTM. Similar to the first method, it starts with segmenting separation lines between building sections and but also extending them to roof planes. Each building is thus divided into sections and each section is divided into roof planes. This process is performed end-to-end by an FCN, which uses a novel depth attention module (DAM) to utilize DSM features effectively and efficiently. The roof surfaces are then converted to polygons. To obtain a 3D model, the height values from the DSM within each roof plane are passed to random sample consensus (RANSAC), which estimates robust plane parameters. The plane parameters are used to calculate the height values at the corners of the roof plane polygons. The experimental evaluation shows that PLANES4LOD2 generates geometrically accurate, topologically consistent and semantically correct LoD-2 models. In comparison with SAT2LOD2, PLANES4LOD2 shows a significantly higher accuracy of 1.06 m in mean average error (MAE), especially in complex scenarios in densely built-up city centres. The baseline method only achieves 2.18 m in MAE. The third method presented in this dissertation is called SAT2BUILDING. It is a method for LoD-2 reconstruction based on image data and DSM, but focuses on satellite rather than aerial image data and does not require an external DTM. Alternatively, an FCN generates the building heights as an additional output. Instead of focusing on the segmentation of separation lines, like PLANES4LOD2, SAT2BUILDING clusters pixels to roof planes by calculating spatial embeddings related to the centre of the roof planes. This is more effective for satellite data with ground sampling distance (GSD) ranging from 0.5 m to 0.7 m, as the separation lines are difficult to recognise. SAT2BUILDING is evaluated on a more rural scenario with simple roof shapes and distances between the buildings and an urban scenario with dense development and complex roof shapes. The metrics show the significantly higher accuracy of SAT2BUILDING compared to three baseline methods in both scenarios. The three newly introduced methods as well as their detailed experimental evaluation and discussion represent a significant contribution to the retrieval of building information and 3D reconstruction of buildings, and enable their future use for a wide range of applications.