Visual Localization of a UAV using Semantic Environment Information and Map Data

Abstract

The use of Unmanned Aerial Vehicles (UAVs) has grown significantly in recent years, driven by technical advancements that have expanded their applications to areas such as rescue missions, delivery systems, and precision agriculture. These applications often require precise real-time localization of the UAV. While most systems rely on Global Navigation Satellite System (GNSS) based localization due to its widespread availability, GNSS signals are susceptible to vulnerabilities such as jamming, spoofing, and multipath propagation. These issues can lead to significant localization errors, making GNSS unreliable in certain scenarios. To overcome these limitations, alternative localization methods must be explored. Visual localization presents a promising approach, leveraging the onboard cameras commonly equipped on UAVs. The method proposed in this work, combines visual data with a reference query database, such as maps, to estimate the UAV’s position. In this work, the open access maps from OpenStreetMaps are used as reference. Using a given segmentation network, segmentation maps are generated from the captured images. From the resulting segmentation maps, Geographic Information System features, such as street and building information, are extracted and subsequently used for estimating the position of the UAV by matching these features against the reference database.