Tower Controller Command Prediction for Future Speech Recognition Applications

Abstract

Air traffic controllers' (ATCos) workload often is a limiting factor for air traffic capacity. Thus, electronic support systems intend to reduce ATCos' workload. Automatic Speech Recognition (ASR) can extract controller command elements from verbal clearances to deliver automatic air traffic control (ATC) system input and avoiding manual input. Assistant Based Speech Recognition (ABSR) systems with high command recognition rates and low error rates have proven to dramatically reduce ATCos' workload and increase capacity as an effect. However, those ABSR systems need accurate hypotheses about expected commands to achieve the necessary performance. Based on the experience with an ATC approach hypotheses generator, a prototypic tower command hypotheses generator (TCHG) was developed to face current and future challenges in the aerodrome environment. Two human-in-the-loop multiple remote tower simulation studies were performed with 13 ATCos from Hungary and Lithuania at DLR Braunschweig. Almost 40 hours of speech with corresponding radar data were recorded for training of the TCHG prediction models in 2017/2018. More than 45 hours of speech and radar data comprising roughly 4,600 voice utterances were recorded in the second simulation campaign for the TCHG evaluation test end of 2018. The TCHG showed operational feasibility with a sufficiently low command prediction error rate of down to 7.3% and low context portion predicted having a sufficiently fast command prediction frequency of once per 120ms to timely deliver the hypotheses to a speech recognition engine. Thus, the next step is to build an integrated ABSR system for the tower environment.