Automatic Speech Recognition in the Cockpit: A Comparative Study of ASR Models for Pilot Communication

Abstract

Automatic Speech Recognition (ASR) has seen significant advances in aviation, particularly in Air Traffic Control (ATC), however intra-cockpit communication between pilots has remained largely unexplored despite its central role in teamwork and decision-making. This paper takes an application-oriented perspective and examines how openly available state-of-the-art ASR models perform when applied to intra-cockpit communication without any domain-specific adaptation. We evaluate OpenAI’s Whisper (Large-v3 and turbo variant), Wav2Vec2-XLSR-53 as a base model with fine-tuned English, German and multilingual versions, and Meta’s Massively Multilingual Speech (MMS) model. Using a dataset of 409 manually transcribed speech segments collected from simulator flights, this paper classifies cockpit communication into six categories and assess performance using Word Error Rate (WER) for each model and category. Results show that Whisper Large consistently achieves the lowest average error rates and demonstrates strong multilingual handling, though it is prone to outliers and occasional hallucinations. Wav2Vec-based models, while less accurate overall, avoid generative errors, with monolingual fine-tuned models working better in language-specific contexts and multilingual variants being able to adapt to code-switching in some cases. The findings highlight trade-offs between consistency, multilingual capability, and computational work, and point to the potential of domain-specific fine-tuning, as this enables improvements in specialized terminology handling. These insights provide a foundation for applying ASR to cockpit communication in both human factors research and future Human-AI Teaming (HAT) applications.