Eye movement indicators for successful failure detection

Abstract

It is becoming increasingly important for pilots and air traffic controllers (ATCs) to be able to detect automation failures in a timely manner. In the context of personnel selection, conventional tests based on behavioural indicators could be complemented by integrating eye-tracking methods. The present study focuses on revealing eye movement parameters that reflect adequate scanning behaviour, which, in turn, predicts accurate failure detection. Eye movements were recorded whilst subjects were monitoring an automated system and reporting failures. Based on predefined areas of interest (AOIs), eye movement parameters were analyzed within different time units around the automation failure. The data suggest that there are differences between the eye movements of operators who detected automation failures and those who missed them. Human operators who successfully detect an automation failure demonstrate time-specific monitoring patterns. These patterns are quantified by parameters such as fixations counts, gaze durations, mean fixation durations, and the total time to first fixation. Depending on the time frame, different eye tracking parameters become relevant for failure detection, thus reflecting the interplay of the diverse cognitive processes involved. The findings are discussed in the context of the personnel selection and training of aviation operatives, as well as ATC incident reporting.