The Federal Aviation Administration (FAA) estimates a 60 million air traffic volume by 2040. However, the available workforce of expert air traffic controllers (ATCs) might not be sufficient to manage this anticipated high traffic volume. Thus, to maintain the same safety standard and service level for air travel, more ATCs will need to be trained quickly. Previous research shows eye tracking technology can be used to enhance the training of the ATC’s by reducing their false alarm rate, thereby helping to mitigate the impact of increasing demand. Methods need to be developed to better understand experts’ eye movement (EM) data so as to incorporate them effectively in ATCs’ training process. However, it’s challenging to analyze ATCs’ EM data for several factors: (i) aircraft representation on radar display (i.e. targets) are dynamic, as their shape and position changes with time; (ii) raw EM data is very complex to visualize, even for the meaningful small duration (e.g. task completion time of 1 min); (iii) in the absence of any predefined order of visual scanning, each ATC employ a variety of scanning strategies to manage traffic, making it challenging to extract relevant patterns that can be taught. To address these aforementioned issues, a threefold framework was developed: (i) a dynamic network-based approach that can map expert ATCs’ EM data to dynamic targets, enabling the representation of visual scanning strategy evolution with time; (ii) a novel density-based clustering method to reduce the inherent complexity of ATCs’ raw EM data to enhance its visualization; (iii) a new modified n-gram based similarity analysis method, to evaluate the consistency and similarity of visual scanning strategies among experts.

Two different experiments were conducted at the FAA Civil Aerospace Medical Institute in Oklahoma City, where EM data of 15 veteran ATCs’ (> 20 years of experience) were collected using eye trackers (Facelab and Tobii eye trackers), while they were controlling a high-fidelity simulated air traffic. The first experiment involved en-route traffic scenario (with aircraft above 18,000 feet) and the second experiment consisted of airport tower traffic (aircraft within 30 miles radius from an airport).

The dynamic network analysis showed three important results: (i) it can be used to effectively represent which are the important targets and how their significance evolves over time, (ii) in dynamic scenarios, having targets having variable time on display, traditional target importance measure (i.e. the number of eye fixations and duration) can be misleading, and (iii) importance measures derived from the network-based approach (e.g. closeness, betweenness) can be used to understand how ATCs’ visual attention moves between targets. The result from the density-based clustering method shows that by controlling its two parameter values(i.e. spatial and temporal approximation), the visualization of the raw EM data can be substantially simplified. This approximate representation can be used for better training purpose where expert ATC’s visual scanning strategy can be visualized with reduced complexity, thereby enhancing the understanding of novices while maintaining its significant pattern (key for visual pattern mining). Moreover, the model parameters enable the decision-maker to incorporate context-dependent factors by adjusting the spatial (in pixel) and temporal (in milliseconds) thresholds used for the visual scanning approximation. The modified n-gram approach allows for twofold similarity analysis of EM data: (i) detecting similar EM patterns due to exact sequential match in which the targets are focused and/or grouped together visually because of several eye fixation transitions among them, and (ii) unearth similar visual scanning behavior which is otherwise small perturbed version of each other that arise as a result of idiosyncrasies of ATCs. Thus, this method is more robust compared to other prevalent approaches which employ strict definitions for similarity that are difficult to empirically observe in real-life scenarios.

To summarize, the three methods developed allow us to apply a comprehensible framework to understand the evolving nature of the visual scanning strategy in complex environments (e.g. air traffic control task) by: (i) by identifying target importance & their evolution; (ii) simplifying visualizing of complex EM strategy for easier comprehension; (iii) evaluating similarity among various visual scanning strategies in dynamic scenarios.