Integration of HoloLens 2 as a Head-Worn Display in a Research Helicopter

Kurzfassung

We present the flight testing and integration of the Microsoft HoloLens 2 as a head-worn display (HWD) in DLR’s research helicopter. Building on its successful use in a helicopter simulator, initial flight tests confirmed its feasibility in a real helicopter. Current tests focused on system optimization, with head tracking identified as the critical component for hologram stability. Since the HoloLens’ inside-out tracking fails in moving vehicles, it was fused with an external infrared tracker, automatically calibrated via an optimization approach adaptable to various trackers and mounting positions. A test pilot with HWD experience rated the system as fully functional, enabling the first successful experiments with holographic Mission Task Elements. Beyond the helicopter, the HoloLens was tested in a car and on a high-speed boat, where holograms remained spatially stable despite high-frequency movements, with a maximum low-frequency error of 0.6° in heading. Static errors depended solely on the external tracker’s quality. These results demonstrate the HoloLens 2’s potential for operational use in dynamic vehicle environments, enhancing immersion. Its mixed-reality features and adaptability proved particularly valuable for rapid research and development across platforms.