My aircraft talks to me: Current developments on voice synthesis as a modality in the cockpit of future fighter aircraft

Abstract

The EPIIC project conducts research on the integration of innovative technologies into the cockpit of the future fighter aircraft. One of those technologies is the synthesis of human voice to generate aural recommendations, warnings and commands. This paper reports on the early stages of development of a voice synthesis module within the EPIIC project. The state-of-the-art on voice warning systems in current fighter jet aircraft is outlined and leads to the introduction of the implemented text-to-speech models, i.e. Bark by suno, SpeechT5 from Microsoft, NVIDIA's FastPitch and MetaVoice by MetaVoiceIO, which are also characterized by a basic assessment of performance parameters as well as a subjective rating from the model developer on the suitability to be used in a cockpit environment. In order to also get the rating of a potential future user, two exploratory technology demonstrations that were held at Airbus Defence & Space (Spain) in 2024 are described. Among other cockpit modalities, the voice synthesis was tested by an active fighter pilot in simulated air warfare scenarios using a fighter cockpit mockup. The comments issued by both the pilot and observers suggest that not necessarily the synthesized voice leaves room for improvement but more the acquisition and processing of relevant data is a key factor for the efficient auditory presentation of information. A brief outlook provides an overview of the way forward in the EPIIC project including possible areas for further improvement of the voice synthesis module.