Vision-based Self-Localization for UAVs using Semantic Features and OpenStreetMap

Abstract

The autonomous operation of an unmanned aerial vehicle (UAV) relies on reliable self-localization, which is typically achieved using global navigation satellite systems (GNSS). However, GNSS data can be unreliable due to effects of space weather phenomena or interference from GNSS jamming. To ensure accurate localization in such conditions, vision-based approaches for UAV positioning offer a potential alternative, though they often come with trade-offs in positioning accuracy or computational efficiency. In this paper, we present a real-time method for vision-based UAV self-localization that achieves GNSS-like accuracy. This approach involves extracting high-level semantic features from captured images and matching them to geo-referenced OpenStreetMap (OSM) data of the flight area. The global location of the UAV is then determined based on the matching results. We also compare and evaluate different metrics for measuring scene similarity to enhance the system's performance. Moreover, we demonstrate that even when OSM data is partially inaccurate, it can still be used to achieve accurate localization. This holds true even with a non-optimal neural network for segmentation and in environments with limited semantic features. The dataset used for evaluation will be made available with publication of this paper.