Towards Robust Multilingual Cockpit ASR with Whisper Fine-Tuning and Waveform Augmentation

Kurzfassung

We present a domain adaptation study of Whisper models for automatic speech recognition (ASR) of cockpit speech. The task is characterized by several significant challenges, including domain-specific aviation vocabulary, code-switching between German and English, short command-like utterances, and acoustic conditions that differ substantially from those represented in the large-scale pretraining data of general ASR models. A corpus of approximately 180 minutes of cockpit recordings was collected and manually annotated by native speakers with an aviation background. The data were split into training and test sets and evaluated using the Whisper model family. The effect of transcript normalization on word error rate (WER) was examined, and a set of normalization schemes was proposed to reduce mismatches between reference and predicted transcripts. The Whisper Large-v3 model was fine-tuned using parameterefficient Low-Rank Adaptation (LoRA), reducing WER from 57.61\% to 24.90\% through fine-tuning and the proposed transcript normalization scheme. To improve robustness under limited data, waveform-level augmentation methods were further applied, including jittering, time warping, window slicing, window warping, scaling, and magnitude warping. Fine-tuning on the augmented training set provided an additional WER reduction of approximately 3–4\% on the held-out test set. These results show that careful normalization, parameter-efficient finetuning, and simple audio augmentation can substantially improve Whisper-based ASR in low-resource, domain-specific cockpit speech settings.