Virtual cockpit instruments displayed on head-worn displays - Capabilities for future cockpit design

Abstract

Head-worn displays (HWDs) have been used on military flight decks for many years. The advancement of this technology - commonly known as augmented reality (AR) - makes it also an interesting choice for the human-machine interface (HMI) design of future civil aircraft. This paper presents an example of how AR glasses can impact new cockpit layouts. We discuss several possibilities of designing and integrating so-called virtual cockpit instruments (VCIs), which are generated by an HWD. The advantages and drawbacks of the proposed HMI are assessed with a human factors study. In recent years, the augmented and virtual reality technology has seen major advancements. The helmets and goggles became more lightweight and comfortable to wear, the monochrome (green) displays were replaced by multi-color screens, and the complex and bulky optical lens systems were substituted by modern waveguide optics. In summary, this evolution leaded to more capable but also more affordable head-worn displays. Together with the improved form factor and comfort, this gives good reason to predict future usage of such AR devices in next-generation flight decks of civil aircraft. The Institute of Flight Guidance at the German Aerospace Center (DLR) has a long history in research on enhanced and synthetic vision systems, which is the classic application of head-up and head-worn AR systems. Motivated by the capabilities of modern HWDs, our current Research is focusing on a holistic cockpit display concept based on AR technology. For this approach, we implemented so-called virtual cockpit instruments on the HWD. As illustrated in Fig. 1, these can be imagined as a virtual counterpart of conventional panel-mounted LCD-displays. The idea of generating VCIs creates several advantages and new possibilities for HMI design. The appearance of conventional flight instruments is tightly coupled to the specifications of the cockpit flat panel screens. This implies that the display symbology is restricted in size, the number of displays is limited, and the location of the display screen is fixed. In contrast, VCIs are very flexible. Size and position can be adapted to the specific display content and to the requirements of the current flight task. Also, the number of virtual screens can be changed dynamically. This means that one can generate additional display areas which can be positioned and sized according to the needs of the pilot and the flight phase. During approach, a pilot can - for instance - open a new virtual screen displaying an approach chart. This "digital kneeboard" can be positioned above the knees like its analog counterpart but also anywhere else around the pilot.In a first evaluation, pilots rated this concept positively. Based on that, a first VCI concept of an obstacle awareness display format for rotorcraft landing and hover tasks was introduced. Our proposed paper further investigates the topic by taking a closer look to the positioning mechanism and the sizing of the VCIs. We present several variants for placing this virtual display within the cockpit. Finally, a comparison of the developed options in terms of flight performance and subjective ratings is performed by means of a human factors study in our helicopter flight simulator. The evaluation shows the potential of augmented reality for future cockpit design and identifies starting points for further research.