Stereo Depth Estimation using Deep Learning: Leveraging Context through Multi-Task Training

Abstract

Dense depth information is vital for robotics applications to fully understand or reconstruct a 3D scene. Recent work has shown that stereo depth estimation through binocular disparity has been successfully cast a learning problem lever- aging convolutional neural networks for a constant surge in performance and accuracy. However, textureless regions, object boundaries and small details still give rise to challenges. The explicit incorporation of semantic knowledge can po- tentially mitigate this problem by providing high-level information specifically for objects and smooth regions. The proposed network architecture derives a com- mon representation for semantic segmentation and disparity estimation through multi-task learning, where the use of an auxiliary task has proven beneficial in terms of learning efficiency and prediction accuracy of the assigned tasks. The training of the disparity estimation model was enabled by synthetically generated data, whereas the resulting disparity output is tested on real images and com- pared in multiple scenarios to a state-of-the-art traditional algorithm.