Integration and use of an AR display in a maritime helicopter simulator

Abstract

Operating a helicopter in off-shore wind parks in DVE from clouds or fog can endanger crew and material due to the presence of unseen obstacles. Utilizing on-board sensors like Lidar or radar one can sense and record obstacles that could be potentially dangerous. One major challenge is then to display the resulting raw sensor data in a way that the crew, especially the pilot can make use of without distracting them from their actual task. Augmented and mixed reality applications are thought to play an important role here. By displaying the data in a see-through helmet mounted display (HMD) the pilot can be made aware of obstacles that are currently obscured by degraded visual conditions or even parts of the own helicopter. This can be accomplished in one HMD. No attention sharing between the outside view and a head-down instrument is necessary. DLR is continuously aiming at testing and evaluating both flight-proof and consumer grade HMDs. One particular widely known system is the Microsoft HoloLens. DLR will integrate this low-cost HMD into their test helicopter ACT/FHS. For this, as a first step a Microsoft HoloLens was integrated into DLR's simulator AVES. In this paper the integration process is detailed. The simulation capabilities are described, especially for conformal, open-loop Lidar sensor data. Furthermore, first concepts of the display format are shown, and strengths and drawbacks of the HoloLens in a cockpit environment are discussed.