A Sensor-AI approach to improve visual odometry in dynamic environments

Abstract

A key component of the interdisciplinary research field Sensor-AI is the close interaction and combination of physical models, data based models and classical approaches in one sensor system, primary for applications that are defined by strict energy requirements. This concept finds practical use cases in many domains. I propose a Sensor-AI based approach that targets to improve visual odometry specifically in uncommon high dynamic environments. It combines classical feature-based visual odometry, which relies on physical models and analytical error propagation, with a data-based model in form of a deep neural network for semantic segmentation. The former is used to automatically generate semantic labels to train the neural network offline, simultaneously for multiple environments, and use its inference output to generate a mask for feature selection online. The proposed method is evaluated on datasets that contain the two dynamic environment elements steam and water, recorded at a volcanic fumarole field, a coast line and a river.