Automated and Precise 3D Building Reconstruction using UAVs

Abstract

In light of the tremendous advances in the fields of unmanned aerial vehicles (UAVs) and imaging sensors in recent years, UAV-photogrammetry has become an essential part of remote sensing methodology. Being more than an alternative to conventional image acquisition platforms, UAV-photogrammetry has revealed novel possibilities and explored a variety of application fields, including the generation of high-quality 3D building models which are of growing importance in the area of 3D city modeling and civil engineering. Nevertheless, practical utilization of UAVs for the task of 3D modeling is still accompanied by various cumbersome activities, such as manual flight planning, deployment of ground control points (GCPs) and manual registration of disconnected 3D models. To this end, this thesis aims to address key challenges in the process of using UAVs for photogrammetric applications and proposes several methods for advancing the state-of-the-art in different stages of UAV-based photogrammetry. Focusing on 3D modeling of buildings, this thesis contributes methods for an automation of the reconstruction process ranging from (i) an accurate image-based multi-modal geo-referencing of acquired images, (ii) an automatic and semantic-aware 3D UAV image acquisition flight planning, (iii) an automatic alignment between individual 3D reconstructions of interior and exterior building models and (iv) a comprehensive investigation of current deep learning-based methods for the task of single-image depth estimation (SIDE), which could contribute to certain areas of image-based 3D building reconstruction. Based on the results of several real-world experiments, the proposed image matching method achieves pixel-level registration accuracies between UAV and multi-modal remote sensing imagery despite significant geometric, radiometric and temporal differences.The model-based 3D path planning method allows for acquiring close-range multi-view stereo-capable image sequences in tightly built-up environments that cover the entire building in a demanded resolution. By incorporating semantic cues into the path generation process, the resulting trajectories are by far more desirable in terms of flight safety by respecting pre-defined restricted and hazardous airspaces such as adjacent buildings or roads. The alignment of individual image-based indoor and outdoor building models is addressed by matching insufficiently overlapping geometric structures, which are shared in both models using 3D line segments as geometric features. A wide variety of experiments on different buildings have verified an accurate registration in centimeter-level accuracy. A comprehensive assessment of current SIDE methods with novel evaluation metrics on a high-quality RGB-D dataset reveals their current suitability for potential practical application fields and emphasizes remaining challenges in this research field. Backed by thorough experimental evaluations confirming the validity of the proposed methods, this thesis marks a step towards an automated, fast, accurate and safe use of UAV photogrammetry.