How to measure monitoring performance of pilots and air traffic controllers

In prior research on the future of aviation it was established that operators will have to work with highly automated systems. Increasing automation will require operators monitoring appropriately (OMA). OMA are expected to demonstrate the use of distinctly different monitoring phases (orientation, anticipation, detection, and recheck). Within these phases, they must grasp in time the relevant information that would enable them to take control should automation fail. The presented study aims at finding appropriate measurements for the identification of OMA on the basis of eye tracking. In order to do this, a normative model of adequate monitoring behavior was designed including the definition of areas of interest. We tested 90 participants who had to monitor a dynamic automatic process, and then take control. In order to decide on suitable eye tracking parameters it was asked which parameters are significantly related to manual control performance. The results show that the suitability of parameters depends on the specific phase of the monitoring process. Gaze durations allow for differentiating between high and low performing subjects during orientation phases. In contrast, relative fixation counts are suitable for predicting monitoring performance during detection phases. In general, the results support the assumption that eye tracking parameters are appropriate for identifying OMA.