Black-Box Universal Adversarial Attack on Automatic Speech Recognition Systems for Maritime Radio Communication Using Evolutionary Strategies

Abstract

This thesis studies the design, implementation, and evaluation of a new universal adversarial attack targeting automatic speech recognition systems in a black-box setting. A genetic algorithm optimizes universal perturbations consisting of short noise bursts that cause mistranscriptions by balancing text similarity (character error rate) and perceptual audio similarity (Mel energy distance) to keep the noise minimally intrusive. Experiments are conducted on the models Wav2Vec 2.0 and OpenAI's Whisper using the standard English Librispeech dataset and a synthetic maritime radio communication dataset that contains more homogeneous data to investigate the attack's performance under varying parameters such as noise volumes and the number of audio files in the training set. We expose vulnerabilities in state-of-the-art ASR systems and the risks of attacks on safety-critical applications, such as maritime radio communication. We demonstrate that our attack is highly successful, and even an attack trained on a single input works universally. Whisper proves to be more robust against these attacks. We find that universal perturbations generalize better when trained on data more similar to the test set. A semantic defense is developed that presents a novel way to detect the attack. To our knowledge, our work represents the first universal black-box attack against ASR models.