3D perspective views on immersive head-worn displays: can exocentric views increase pilot situational awareness?

Kurzfassung

A pilot's out-the-window view is heavily restricted by non-transparent parts of the airframe. Pilots are hardly able to see the surroundings and threats below, above, and behind them. Moreover, the amount of usable outside visual cues is often drastically decreased, for instance due to fog, heavy precipitation or darkness. As the outside vision constitutes an important information source for helicopter pilots, it is highly desirable to develop solutions that enable operations when the natural vision does not provide enough information to safely fly the aircraft. State-of-the-art solutions gather information about the environment by fusing data from various aircraft-mounted sensors (e.g. radar, lidar) with information from databases. The resulting picture of the environment is superior to the unaided human vision under many visual conditions like darkness or fog. To convey the gathered data to the pilot, two types of displays are used in today's cockpits: conventional panel-mounted displays (PMD) and see-through head-up or helmet-mounted displays (HUD, HMD). The former offer a full color representation and give the display designer great freedom in the way flight guidance information is presented. However, such synthetic or enhanced vision displays can only show a down-scaled projection of the three-dimensional environment on rather small flat panel screens. Further, they tie the pilot's head down to the instrument panel. Head-up and head-worn displays, on the other hand, allow the pilots to focus their eyes out of the cockpit as they superimpose the real-world view with symbology providing additional cues. Nevertheless, these augmented reality solutions also suffer from a number of weaknesses, for instance limited field of view, brightness issues, or lack of color. Our solution combines advantages of both worlds -- 3D perspective head-down and visual-conformal head-up/helmet-mounted displays -- by using a non-transparent head-worn display. The chosen display type enables us to present a full-color, 3D perspective view without scaling the picture down to a flat panel screen. Via head-tracking system, the viewing direction can be coupled to the pilot's head movements implying that the pilots can naturally look around the displayed scene just by turning their heads to the point of interest. They can virtually look below, behind, and above their own aircraft -- areas they cannot see from a conventional cockpit. Further, exocentric perspectives can be realized. This means that the pilot's viewpoint is moved out of the cockpit, for instance to a location behind and above the own aircraft. Three-dimensional ego- and exocentric display perspectives have been thoroughly discussed in research on synthetic vision navigation and primary flight displays. Costs and benefits of different viewpoints and perspectives are well-documented. However, previous research on exocentric views deals with head-down display representations, whereas the proposed approach covers head-tracked, head-worn displays. Moreover, this work places the focus on helicopter-specific tasks like hover and land in close proximity to obstacles. This paper presents the results of a simulator study conducted to evaluate different aspects of the proposed concept. The experiment is carried out in the VR flight simulator shown in Figure 1. Emphasis is placed on comparing several 3D perspective views in terms of their impact on pilot performance, spatial awareness, and workload. Our egocentric view shows the scene as it looks from the pilot's seat. However, with the difference that our synthetic view offers a full 360° field of regard, which is not restricted to window areas as it is the case in a conventional cockpit. Exocentric views can take various forms; in this work different viewpoints and frames of reference are compared. Furthermore, the addition of artificial cues like drop lines or other scene-linked symbology to enhance the information content is discussed. Finally, a combination of multiple complementing 3D views could provide a better picture of the situation. The results of this paper are applied to our current research on all-weather helicopter operations. Nevertheless, the findings regarding 3D perspective views are also relevant for the development of future remote piloting solutions or external vision systems for aircraft with highly restricted cockpit window areas, for instance armored military aircraft or future hypersonic airplanes.