Bousdar Ahmed, Dina und Diez, Luis Enrique und Munoz Diaz-Ropero, Estefania und García Domínguez, Juan Jesús (2019) A survey on test and evaluation methodologies of pedestrian localization systems. IEEE Sensors Journal. IEEE - Institute of Electrical and Electronics Engineers. doi: 10.1109/jsen.2019.2939592. ISSN 1530-437X.
![]() |
PDF
- Nur DLR-intern zugänglich
- Postprintversion (akzeptierte Manuskriptversion)
452kB |
![]() |
PDF
- Nur DLR-intern zugänglich
- Verlagsversion (veröffentlichte Fassung)
2MB |
Kurzfassung
The first formal attempt to standardize the test and evaluation of localization systems was the ISO/IEC 18305 standard. As reviewed in this paper, the ISO/IEC 18305 provides guidelines to evaluate an indoor localization system as a black box. Such evaluation is useful for end users, but not fully useful for either developers or researchers. As a result, both developers and researchers use their own evaluation methodologies, which have a set of weaknesses. Our objective is to compare the standard to the evaluation methodologies implemented by developers. We focus on pedestrian localization systems because they are the most numerous. In this work, we survey both competitions and individual works, between 2008-2018, with the ISO/IEC 18305 standard as a guideline. That is, we have reviewed the evaluation methodologies regarding the different aspects defined in the standard. We finalize with a summary of the shortcomings of the evaluation methodologies implemented by the developers, as well as the shortcomings of the standard. Our article ends with a summary of open issues that need to be addressed in order to foster the development of evaluation methodologies of localization systems. The main conclusion of our work is that the developers of localization systems are in need of guidelines to design evaluation methodologies, especially for the evaluation of system components. The standard fails to satisfy such requirement by proposing an evaluation strategy that focuses more on the end user than the developer.
elib-URL des Eintrags: | https://elib.dlr.de/129504/ | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Zeitschriftenbeitrag | ||||||||||||||||||||
Titel: | A survey on test and evaluation methodologies of pedestrian localization systems | ||||||||||||||||||||
Autoren: |
| ||||||||||||||||||||
Datum: | 5 September 2019 | ||||||||||||||||||||
Erschienen in: | IEEE Sensors Journal | ||||||||||||||||||||
Referierte Publikation: | Ja | ||||||||||||||||||||
Open Access: | Nein | ||||||||||||||||||||
Gold Open Access: | Nein | ||||||||||||||||||||
In SCOPUS: | Ja | ||||||||||||||||||||
In ISI Web of Science: | Ja | ||||||||||||||||||||
DOI: | 10.1109/jsen.2019.2939592 | ||||||||||||||||||||
Verlag: | IEEE - Institute of Electrical and Electronics Engineers | ||||||||||||||||||||
ISSN: | 1530-437X | ||||||||||||||||||||
Status: | veröffentlicht | ||||||||||||||||||||
Stichwörter: | standardization, guidelines, performance, navigation, methodologies, positioning, review | ||||||||||||||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||||||||||||||
HGF - Programm: | Verkehr | ||||||||||||||||||||
HGF - Programmthema: | Straßenverkehr | ||||||||||||||||||||
DLR - Schwerpunkt: | Verkehr | ||||||||||||||||||||
DLR - Forschungsgebiet: | V ST Straßenverkehr | ||||||||||||||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | V - NGC KoFiF (alt) | ||||||||||||||||||||
Standort: | Oberpfaffenhofen | ||||||||||||||||||||
Institute & Einrichtungen: | Institut für Kommunikation und Navigation > Nachrichtensysteme | ||||||||||||||||||||
Hinterlegt von: | Bousdar Ahmed, Dina | ||||||||||||||||||||
Hinterlegt am: | 15 Okt 2019 12:45 | ||||||||||||||||||||
Letzte Änderung: | 24 Okt 2023 12:00 |
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags