Efficient Large-Scale Stereo Reconstruction using Variational Methods

Kurzfassung

This thesis investigates the use of convex variational methods for depth reconstruction from optical imagery and fusion of multiple depth maps into combined depth maps with higher accuracy. Dense depth reconstruction from two or more camera views are an important subject of research in computer vision, since measurement density is much higher than other depth sensing techniques, namely active depth sensing via infrared pattern projection or Lidar and Radar based techniques - even though the latter ones are more accurate and robust in depth. Other advantages of cameras are their low costs and low power consumption due to their passive sensing principle. Approaches are ranging from autonomous driving cars, obstacle avoidance or surveying UAVs up to detailed reconstruction of remote terrains using spaceborne imagery. In particular, we propose a fast algorithm for high-accuracy large-scale outdoor dense stereo reconstruction. To this end, we present a structure-adaptive second-order Total Generalized Variation (TGV) regularization which facilitates the emergence of planar structures by enhancing the discontinuities along building facades. Instead of solving the arising optimization problem by a coarse-to-fine approach, we propose a quadratic relaxation approach which is solved by an augmented Lagrangian method. This technique allows for capturing large displacements and fine structures simultaneously. For the application in autonomous driving, we further present an algorithm for dense and direct large-scale visual SLAM that runs in real-time on a commodity notebook. We developed a fast variational dense 3D reconstruction algorithm which robustly integrates data terms from multiple images thus enhancing quality of the Image matching. An additional property of this variational reconstruction framework is the ability to integrate sparse depth priors (e.g. from RGB-D sensors or LiDAR data) into the early stages of the visual depth reconstruction, leading to an implicit sensor fusion scheme for a variable number of heterogeneous depth sensors. Embedded into a keyframe-based SLAM framework, this results in a memory efficient representation of the scene and therefore (in combination with loop-closure detection and pose tracking via direct image alignment) enables us to densely reconstruct large scenes in real-time. Finally, applied to space-borne remote sensing, we present an algorithm for robustly fusing digital surface models (DSM) with different ground sampling distances and confidences, using explicit surface priors to obtain locally smooth surface models. The optimization using L1 based differences between the separate DSMs and incorporating local smoothness constraints is also inherently able to include weights for the input data, therefore allowing to easily integrate invalid areas, fuse multiresolution DSMs and to weigh the input data.