Development of an Augmented Vision Videopanorama Human-Machine Interface for Remote Airport Tower Operation

Abstract

In this paper the design process, development and initial version of an augmented vision video panorama system for remote airport tower operation (RTO) is described. RTO aims at remote control of small air-ports or of movement areas not directly visible for the controller. The work is performed within the project Remote Airport Tower Operation Research (RapTOr) of the German Aerospace Center which followed an initial concept study started in 2002 [1][2][3]. RTO is the first step on the way to the Virtual Tower (ViTo) as long term goal for replacing tower buildings also on larger airports. The direct view out of the tower windows is of central importance for surface traffic control under the present day working conditions of tower and apron controllers [4][5]. Consequently it is assumed that un-der the guideline of human centered automation, the real time reconstruction of the direct far view with su-perimposition (augmentation) of traffic relevant data will greatly improve the transition process to the new work environment and make it acceptable to the user. The augmented vision video panorama system serves as the core element of the RTO controller's hu-man machine interface (HMI). The RTO workplace is located in a remotely located control center, e.g. a tower at a medium size airport or a tower center for the remote control of several small airports. The setup and functions of the high resolution video panorama system with integrated pan-tilt zoom camera (PTZ) at Braun-schweig research airport is described. It provides the framework for integrating approach radar and ground traffic data, real time image processing and a full day replay possibility of a complete 180° - panorama. Concerning the design process we describe how the results of a formal tower work analysis (formal cogni-tive ressource model, FCR [7]) are transferred into a cognitive operator model for simulating the RTO con-troller work processes. The simulation tool supports the design process of the RTO HMI. The design and development of the experimental RTO HMI is accompanied by a cognitive work and task analysis (CWA) of the presently existing work environment and decision processes in a medium size airport. It provides the input data for a specific decision model and for the computer simulation of controller decision making in the new tower work environment with additional RTO HMI. The CWA is based on a formal procedure suggested by Vicente [6], separating the analysis into five areas: work domain analysis, con-trol task analysis, strategy analysis, analysis of social organisation and cooperation, and operator competency analysis. The formal mathematical background of the decision making simulation is a colored Petri net model of the work and traffic processes [7] [8] [9]. Simulations support the investigation of the controller's work process with respect to HMI design alternatives, without the necessity of actually realizing each of the alternative solutions. An important question to be in-vestigated by simulating the work processes refers to the number of operators: can the additional control of the remote airport be performed by the existing con-troller team or is an additional RTO controller re-quired? This question of course is closely related to the design of the new RTO workplace and the integration into the existing tower environment as well as possible new work organisation and training procedures. Augmentation of the reconstructed video pano-rama (augmented tower vision (ATV)) e.g. by integra-tion of approach radar information, aircraft labels with object tracking and weather data, appears to be an es-sential aspect for creating an optimized RTO work situation. It may also help reducing the RTO control-lers workplace size, i.e. number of monitors and inter-action devices, in order to fit into the conventional me-dium size tower work environment. However, ATV also rises questions adressing controllers attention and perception processes. Potential problems like atten-tional tunneling and involuntary switching of percep-tion [10] [11] have to be considered for the design of the augmented vision video panorama. Initial field tri-als as well as laboratory experiments with superim-posed information on the far view using transparent displays have been performed in order to adress the question of acceptance of augmented vision systems by controllers and the aspect of head down time reduction for reducing the number of monitors [12]. Previous results reported in the literature showed that the ex-pected advantage of reduced visual scanning between the head-down instruments and the far domain is not stable under all circumstances and may be counteracted by effects such as attention capture and visual interfer-ence [10] [11] [13]. A high resolution video panorama system has been set up at Brauschweig research airport as experi-mental environment for investigation of different technical and psychological aspects of the video based RTO – HMI and for the development of a demonstra-tor. The photos of Fig. 1 show the panorama camera system (1600 x 1200 pixel, 14 bit dynamic range) in-cluding the remotely controlled pan-tilt zoom (PTZ) camera as basic sensor system and the panorama and PTZ monitor system as main component of the RTO controller HMI.Digital data transmission between camera and display systems is performed via GBit ethernet employing high performance PC clusters at camera and display loca-tions for compression and decompression. A system for real time image processing with the un-compressed video data using FPGA implementation of the image processing software is under construction for operating in parallel to the panorama system. Initial results of offline processing exhibit the quality of background subtraction algorithms for extracting the moving objects. Initial results of the system performance obtained to-wards the end of 2006 with systematic flight tests (landing, takeoff, ground movement) will be reported.