Orientation and Dense Reconstruction of Unordered Terrestrial and Aerial Wide Baseline Image Sets

Abstract

In this paper we present an approach for detailed and precise automatic dense 3D reconstruction using digital images from consumer cameras. The major difference between our work and many others is that we are able to deal with wide-baseline image sets. We have combined and improved several methods, e.g., least squares matching, RANSAC (Fischler & Bolles 1981), scale-space maxima (Lindeberg 1994), and bundle adjustment, for robust matching and parameter estimation. Point correspondences and the five-point algorithm (Nistér 05) lead us to the relative orientation. Due to our robust matching method it is possible to orient images under much more unfavorable conditions, for instance concerning illumination changes or scale differences, than it is feasible with often used operators such as SIFT (Lowe 2004). Based on the highly reliable relative orientation we can calculate the absolute orientation from unreliable and imprecise GPS data of low cost sensors, e.g., in a GPS camera, also in areas with strong occlusion, e.g., cities (Bartelsen & Mayer 2010). For dense reconstruction we use the results of our orientation as input for Semi-Global Matching (SGM) (Hirschmueller 2008). Although SGM has been developed for short baseline image sets, very good depth estimates are also possible for wider baselines, due to our precise relative orientation (cf. attached figure, right). This results in dense depth images, which can be fused into a 2.5D model for eliminating the high redundancy of the highly overlapping depth images. However, some applications require full 3D models. Currently, we do not see a satisfying approach for full, high resolution 3D reconstruction of large models, that maintains all the details that are available in the depth images. A solution to this problem is part of our current work. Some early results are presented in this paper. With very small unmanned aerial systems (Micro UAS) it is possible to acquire images which have a similar perspective as terrestrial images and can thus be combined with them. Such a combination is useful for an almost complete 3D reconstruction of settlements (cf. attached figure, left). We combined several hundreds of aerial and ground images and received a detailed model of a large urban area. In principle we are able to deal with unordered image sets by employing the approach of (Yao & Cham 2007). Yet, many image pairs, which can be oriented by our robust matching method, cannot be found due to the limited capability of the employed fast matching method. Therefore, while a combination of aerial and ground images is possible, in this case the detection of overlapping images has to be done manually.