Using Mixed Reality in Simulator Based Flight Training for Helicopters

Kurzfassung

The integration of Mixed Reality (MR) technology into flight training is a promising solution to meet the growing demand for pilots, as the aviation industry seeks more flexible and cost-effective training solutions. This dissertation investigates the suitability of MR headsets in helicopter pilot training, specifically focusing on whether MR can enhance or complement traditional simulators. To address these objectives, three studies were conducted using two distinct helicopter simulators: two studies in a basic dual pilot station simulator with TV screens and one study in the fully equipped EC135 simulator with a projection dome.

The first study compared a monitor-based headset (Varjo XR-3 Focal Edition) with a Mixed Reality (MR) and Virtual Reality (VR) configuration against a see-through headset (JVC HMD-VS1D) with Augmented Reality (AR) configuration and conventional TV displays as the visual system. The results showed that while flight performance was similar across all systems, the Varjo MR configuration yielded higher presence ratings with negligible simulation sickness, leading to the selection of the Varjo headset with the MR configuration for further testing.

In the second study, the Varjo headset was integrated into the fully equipped EC135 cockpit. This study demonstrated the feasibility of a hybrid MR approach using chroma key and cutouts to minimize visual artifacts, enabling successful cockpit interaction and malfunction procedures, however, challenges such as restricted Field of View (FOV) limitations persisted.

A comparative analysis of the first two studies’ results found a positive correlation between presence and flight performance. Additionally, the analysis identified distinct gaze behaviors in MR setups, where pilots tended to focus more on external visual cues than with other visual configurations. The third study focused on the evaluation of proposed standardized flight tasks based on annual proficiency checks to assess new visual systems, such as MR headsets, for helicopter pilot training in industrial settings and the applicability of the used assessment methods.

The findings show that MR headsets can serve as suitable visual systems for helicopter pilot training. While MR currently falls short of the presence and comfort provided by projection domes, it offers higher presence and comparable flight performance to basic mock-up simulators. Furthermore, MR technology offers significant advantages in terms of flexibility and compactness, allowing for easier deployment and increased accessibility of training facilities. As MR technology advances, it holds the potential to enhance or complement training conducted in full-flight simulators.