3D Urban Areas Reconstruction from High Resolution 2D Images

Abstract

The reconstruction of detailed 3D point clouds of urban buildings from single-view images remains challenging due to limitations in existing methods, which often focus on rooftops while neglecting walls and the ground. The lack of comprehensive datasets containing complete 3D point clouds and the difficulty in obtaining accurate camera pose information from single-view images further complicate the process. To address these challenges, we propose a novel approach that reconstructs detailed 3D point clouds of urban buildings, capturing the full structure, including rooftops, walls, and the ground, for a more comprehensive representation. Our method leverages a generative AI diffusion model guided by edge-aware features, such as binary masks and Sobel edge maps, to progressively refine geometric details. These features enable the model to better capture architectural contours, improving the accuracy and precision of the 3D reconstruction. A key contribution of our work is the creation of a custom dataset to address the scarcity of comprehensive 3D data. This dataset includes complete 3D point clouds and camera pose information, predicted directly from single-view images using our methodology. By incorporating these predicted camera parameters, the dataset ensures accurate alignment of features onto the 3D point cloud, providing a robust foundation for model training and evaluation. The results demonstrate that our method outperforms existing techniques, achieving highly accurate and detailed 3D reconstructions of urban buildings, with generalizability proven on another processed data from single images of Tallin City, Estonia.