Real Time Validation of Online Situation Awareness Questionnaires in Simulated Approach Air Traffic Control

Abstract

Measuring Situation Awareness (SAw) to evaluate an operator's ability to handle complex dynamic situations and the use of assistance systems have become a standard approach in Human Factors research. Ideally, the operators should be supported by enabling and disabling assistance systems depending on their SAw. On the one hand, if the situation's complexity increases and therefore SAw is likely to be reduced, additional systems may help to prevent overextension by taking control over the specific task or task components. On the other hand, there has been evidence that high levels of automation may reduce the ability to intervene in a timely manner if needed due to mental underload. Adjusting the usage of assistance systems based on the operator's SAw may help to overcome both limitations. However, existing measurement tools for SAw require post-simulation analysis. This way it is not possible to make decisions based on the operator's SAw in parallel with the situation at hand. Software capable of analyzing the current state of a given situation has been developed to allow real time assessment of SAw. This software was designed to measure SAw of approach air traffic controllers in the real time NLR ATM Research Simulator. SAw was measured by presenting questionnaires during three different scenarios. Before an item was presented, the simulation's log files were analyzed to provide the software with the correct answer. This way, validating responses and evaluating SAw immediately was possible. In a first study, 57 non-expert subjects were presented with online probe questionnaires in real time simulated approach air traffic control scenarios. It was found that the software was able to measure SAw in real time. In the future, such systems could be used to make decisions about the need for further assistance while the situation is still happening. This way, operators would only get the necessary amount of assistance without reducing their work to passive monitoring.