Readback Error Detection by Automatic Speech Recognition and Understanding - Results of HAAWAII project for Isavia’s Enroute Airspace

Abstract

One of the crucial tasks of an air traffic controller (ATCo) is to evaluate pilot readbacks and to react in case of errors. Undetected readback errors, when not corrected by ATCos, can have a dramatic impact on air traffic management (ATM) safety. Although they seldomly occur, the benefits of even one prevented incident due to automatic readback error detection justifies the efforts. The HAAWAII uses automatic speech recognition and understanding (ASRU) to support the air traffic controller (ATCo) in this critical task. This paper presents two approaches for readback error detection: a data driven approach based on machine learning and a rule-based approach. The combination of both detects 81% of the readback error use cases on real-life voice recordings from Isavia’s airspace. Proof-of-concept trial with six ATCos from Isavia producing artificial, but challenging read back error use cases resulted in a false alarm rate of 11% and a readback error detection rate of 80%. These results are based on Word Error Rates (WER) of 5% for ATCos and of 10% for pilots respectively