DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

A survey on test and evaluation methodologies of pedestrian localization systems

Bousdar Ahmed, Dina and Diez, Luis Enrique and Munoz Diaz-Ropero, Estefania and García Domínguez, Juan Jesús (2020) A survey on test and evaluation methodologies of pedestrian localization systems. IEEE Sensors Journal. IEEE - Institute of Electrical and Electronics Engineers. doi: 10.1109/jsen.2019.2939592. ISSN 1530-437X.

[img] PDF - Only accessible within DLR - Postprint version (accepted manuscript)
[img] PDF - Only accessible within DLR - Published version


The first formal attempt to standardize the test and evaluation of localization systems was the ISO/IEC 18305 standard. As reviewed in this paper, the ISO/IEC 18305 provides guidelines to evaluate an indoor localization system as a black box. Such evaluation is useful for end users, but not fully useful for either developers or researchers. As a result, both developers and researchers use their own evaluation methodologies, which have a set of weaknesses. Our objective is to compare the standard to the evaluation methodologies implemented by developers. We focus on pedestrian localization systems because they are the most numerous. In this work, we survey both competitions and individual works, between 2008-2018, with the ISO/IEC 18305 standard as a guideline. That is, we have reviewed the evaluation methodologies regarding the different aspects defined in the standard. We finalize with a summary of the shortcomings of the evaluation methodologies implemented by the developers, as well as the shortcomings of the standard. Our article ends with a summary of open issues that need to be addressed in order to foster the development of evaluation methodologies of localization systems. The main conclusion of our work is that the developers of localization systems are in need of guidelines to design evaluation methodologies, especially for the evaluation of system components. The standard fails to satisfy such requirement by proposing an evaluation strategy that focuses more on the end user than the developer.

Item URL in elib:https://elib.dlr.de/129504/
Document Type:Article
Title:A survey on test and evaluation methodologies of pedestrian localization systems
AuthorsInstitution or Email of AuthorsAuthor's ORCID iD
Bousdar Ahmed, DinaDina.BousdarAhmed (at) dlr.deUNSPECIFIED
Diez, Luis Enriqueluis.enrique.diez (at) deusto.esUNSPECIFIED
Munoz Diaz-Ropero, EstefaniaEstefania.Munoz (at) dlr.deUNSPECIFIED
García Domínguez, Juan Jesúsjjesus.garcia (at) uah.esUNSPECIFIED
Journal or Publication Title:IEEE Sensors Journal
Refereed publication:Yes
Open Access:No
Gold Open Access:No
In ISI Web of Science:Yes
DOI :10.1109/jsen.2019.2939592
Publisher:IEEE - Institute of Electrical and Electronics Engineers
Keywords:standardization, guidelines, performance, navigation, methodologies, positioning, review
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Transport
HGF - Program Themes:Road Transport
DLR - Research area:Transport
DLR - Program:V ST Straßenverkehr
DLR - Research theme (Project):V - NGC KoFiF
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Communication and Navigation > Communications Systems
Deposited By: Bousdar Ahmed, Dina
Deposited On:15 Oct 2019 12:45
Last Modified:15 Jul 2021 16:26

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.