Single-View Depth from Focused Plenoptic Cameras

Kurzfassung

In recent years, the research progress in computer vision has boosted the capabilities of machines for interpreting visual data, thereby expanding the complexity and range of tasks that robots could perform in fields such as autonomous driving, medicine, and industrial automation. A principal facet of computer vision is depth estimation, crucial for enabling robots to perceive, navigate, and interact with their environment in an effective and safe manner. Traditional setups, like stereo or multi-camera, face challenges such as calibration intricacies and computational and hardware complexity. Further, their accuracy is limited by the baseline between the cameras. Monocular depth estimation, thus using a single camera, offers a more compact alternative but is however limited by the unobservability of the scale. Light field imaging technologies represent a promising solution to the above issues by capturing both the intensity and direction of light rays not only through the main lens, but also through a large number of microlenses placed within the camera. By these means, depth in front of the camera can be measured owing to depth-dependent refraction at the main lens. Despite their potential, there are limited studies exploring their application to single-view dense depth estimation. This scarcity can be attributed to several factors. The technology remains relatively costly and inaccessible for its widespread adoption, leading to a lack of datasets suitable for training deep neural networks. As a consequence, few projects have used light field imaging for depth estimation, and existing efforts often rely on outdated iterations of the technology. Furthermore, the lack of an open-source geometrical model impedes the development of model-based estimation. This thesis explores the potential of focused plenoptic cameras for single-view depth estimation using learning-based methods. The proposed approach integrates techniques from image processing, deep learning, and scale alignment achieved through foundational models and robust statistics, to generate dense metric depth maps. To support this approach, a novel real-world dataset of light field images with stereo depth labels was generated, addressing a current gap in existing resources. Experimental results demonstrate that the developed pipeline can reliably produce accurate metric depth predictions, setting a foundation for further research in this domain.