Bridging Classical and Learned Stereo Vision Approaches: A Comparative Study for Dense, Passive Stereo Depth Estimation

Kurzfassung

Depth perception is a core component across multiple applications, including autonomous systems, augmented reality, and even medical systems. Solutions often need to be fast, dense, and accurate at the same time, while still being affordable for consumer hardware. Passive Dense Stereo Matching has been one of the most used methods in such scenarios, as the passive sensors are cheap. Especially with the rising market share of NVIDIA Jetson boards, demand and research will continue to grow. In the last decade, research in stereo vision has shifted from traditional methods based on handcrafted algorithms and features to deep learning. Those methods are currently mostly evaluated on fully dense images and must predict for every pixel. They are also mostly evaluated on performant hardware, which may not be suitable for many real-world applications. While models get better and better at predicting parts of images, which only one camera could actually see, this is still an unwanted behavior in a robotic setup in a critical environment. There, models should only predict points where they are confident that they can give accurate results. In this thesis, we aim to fill those gaps. We evaluate 27 different stereo configurations, spanning both learned and traditional methods, in difficult environments with large baselines or calibration errors. We compare their results and runtimes on edge devices. We also use confidence measures on learned methods to give fair comparisons to the SGM algorithm, which filters out points where it is not confident. To this end, we propose new confidence measures that operate directly on iterative-refinement-based learned models without additional training, thereby covering most recently published stereo methods. With those confidence methods, models can distinguish between occlusions and difficult-matching areas.