elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

How to Measure Speech Recognition Performance in the Air Traffic Control Domain? The Word Error Rate is only half of the truth

Helmke, Hartmut and Shetty, Shruthi and Kleinert, Matthias and Ohneiser, Oliver and Prasad, Amrutha and Motlice, Petr and Cerna, Aneta and Windisch, Christian (2021) How to Measure Speech Recognition Performance in the Air Traffic Control Domain? The Word Error Rate is only half of the truth. Interspeech 2021, 30.08.2021 - 03.09.2021, Brno, Tschechien.

[img] PDF
267kB

Abstract

Applying Automatic Speech Recognition (ASR) in the domain of analogue voice communication between air traffic controllers (ATCo) and pilots has more end user requirements than just transforming spoken words into text. It is useless, when word recognition is perfect, as long as the semantic interpretation is wrong. For an ATCo it is of no importance if the words of greeting are correctly recognized. A wrong recognition of a greeting should, however, not disturb the correct recognition of e.g. a “descend” command. Recently, 14 European partners from Air Traffic Management (ATM) domain have agreed on a common set of rules, i.e., an ontology on how to annotate the speech utterance of an ATCo. This paper first extends the ontology to pilot utterances and then compares different ASR implementations on semantic level by introducing command recognition, command recognition error, and command rejection rates. The implementation used in this paper achieves a command recognition rate better than 94% for Prague Approach, even when WER is above 2.5%

Item URL in elib:https://elib.dlr.de/145465/
Document Type:Conference or Workshop Item (Speech)
Title:How to Measure Speech Recognition Performance in the Air Traffic Control Domain? The Word Error Rate is only half of the truth
Authors:
AuthorsInstitution or Email of AuthorsAuthor's ORCID iD
Helmke, HartmutHartmut.Helmke (at) dlr.deUNSPECIFIED
Shetty, Shruthishruthi.shetty (at) dlr.deUNSPECIFIED
Kleinert, MatthiasMatthias.Kleinert (at) dlr.deUNSPECIFIED
Ohneiser, OliverOliver.Ohneiser (at) dlr.dehttps://orcid.org/0000-0002-5411-691X
Prasad, Amruthaamrutha.prasad (at) idiap.chUNSPECIFIED
Motlice, Petrpetr.motlicek (at) idiap.chUNSPECIFIED
Cerna, AnetaAneta.Cerna (at) ans.czUNSPECIFIED
Windisch, Christianchristian.windisch (at) austrocontrol.atlUNSPECIFIED
Date:2021
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In SCOPUS:No
In ISI Web of Science:No
Status:Published
Keywords:word error rate, command recognition rate, language understanding, air traffic control, ATC
Event Title:Interspeech 2021
Event Location:Brno, Tschechien
Event Type:international Conference
Event Dates:30.08.2021 - 03.09.2021
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Aeronautics
HGF - Program Themes:other
DLR - Research area:Aeronautics
DLR - Program:L - no assignment
DLR - Research theme (Project):L - Managementaufgaben Luftfahrt
Location: Braunschweig
Institutes and Institutions:Institute of Flight Guidance > Controller Assistance
Deposited By: Diederich, Kerstin
Deposited On:15 Nov 2021 07:30
Last Modified:15 Nov 2021 07:30

Repository Staff Only: item control page

Browse
Search
Help & Contact
Information
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.