Workload and Performance in FOPA: A Strategic Planning Interface for Air Traffic Control

The Flight Organizer Planning Aid (FOPA) interface was designed to aid strategic planning in air traffic control. In particular, FOPA was designed to address difficulties in solving en route sequencing problems (combining multiple streams of traffic into a single stream heading for a destination). We compared the planning performance of 12 full performance level air traffic controllers using either the FOPA interface or normal flight progress strips. Planning performance was significantly better when using FOPA; subjective workload was also reduced. The results indicate that a key advantage of FOPA lies in its dynamic linkage between the flight organizer screen where aircraft tokens can be categorized into color-coded blocks and their matching spatial representation on the radar.


INTRODUCTION
En route air traffic control involves the separation of aircraft and the achievement of an expeditious flow of air traffic through the assigned sector. Currently, controllers have available four sources of information that are used to help them move aircraft most effectively: the radar screen, paper flight progress strips, the computer readout device (CRD), and radio input from pilots. All information has to be integrated in order to control traffic and to issue appropriate commands.
The purpose of the present study was to examine one important aspect of air traffic control planning: the ability of a new computer interface (the Flight Organizer Planning Aid, or FOPA) to aid contiollers' strategic planning. Canning et al. (1999) have previously described the interface at length; in the present article, we report the results of an experiment testing FOPA's effect on planning performance and workload.
Research in the domain of military planning has identified two general types of planning: strategic planning, which is done prior to battle, and tactical planning, which is done during battle. This distinction between strategic and tactical planning can be applied to air traffic control. Specifically, tactical plans are required for the resolution of immediate conflicts between a small number (2 -3) of aircraft and are assumed to have a relatively short execution time (a few minutes). Klein (I 989) proposed the recognitionprimed decision (RPD) model to account for tactical decision making or planning. The RPD model assumes that tactical decisions are closely tied to the perceptual processes of problem identification.
Strategic plans involve multiple aircraft over a relatively long period of time (up to 20 minutes). Strategic planning involves a higher level of planning behavior, characterized by the prioritization and anticipation of future tasks and multiple conflict resolution. Despite the fact that strategic planning seems quite different than tactical planning, little research has been done on it (for an exception involving plan generation, see Hayes-Roth & Hayes-Roth, 1979). Dougherty, Gronlund, Canning, Durso and Mills (1999) identified the task of sequencing aircraft (combining different streams of air traffic into one single stream with special separation requirements) as a situation that involves strategic planning. Consequently, we utilized sequencing problems in our investigation of strategic planning.
In our experiment, two controllers worked the aircraft in a sector. The planner (our participant) was responsible for deciding the sequence of the aircraft, and the tactician (our subject matter expert, or SME) was responsible for implementing that plan and keeping the aircraft separated. The division of labor is similar to what is done at Eurocontrol (Bmggen, 1998) and is also being discussed for the US air traffic control system (Lather & Walker, 1995). We designed FOPA to assist the strategic planning of a controller who is little involved in the tactics of conflict resolution.

Description of FOPA
The FOPA interface consists oftwo 2l-inch color screens that are electronically linked together. The left screen serves as the radar screen ( Figure 1) and contains much of the information a controller normally sees on a radar display. The right-hand screen is the flight organizer, and it contains electronic tokens analogous to flight progress strips. These tokens maintain and enhance the functionality of the paper strips. Our prior work showed that paper flight progress strips were important for strategic planning , and hence we wanted to maintain their functionality in our new interface. The aircraft tokens can be arranged freely into sequencing blocks on the screen. Traffic sequences of aircraft that are grouped together because they are being sequenced to the same destination airport have the same color and are thereby distinguished from other sequences. Finally, each aircraft token on the flight organizer screen is linked to its aircraft representation on the radar screen.
Additionally, FOPA provides several functions that are not available in a regular air traffic control environment. Traffic at specific altitudes can be filtered out using altitude sliders. The cumulative distance between any number of freely chosen points can be displayed. On the flight organizer screen, the flight tokens in each sequence block can be automatically sorted according to time or distance to a freely chosen point, e.g., the sector boundary. Handling of flight tokens on the flight organizer screen is supported so that controllers can move several tokens at the same time. air traffic controllers from the FAA Academy in Oklahoma City participated in the experiment. Participants were trained to use FOPA in two sessions that were held on separate days prior to the experiment. The air traffic sc&mrios were developed by an instructor at the Academy and had approximately equal traffic complexity. The scenarios involved a training sector that was wellknown to all participants.
In the experiment, participants were asked to develop a sequence for a scenario that was presented in a paused mode. The scenario did not begin running until a plan was developed and communicated to a second controller, the tactician sitting adjacent to the planner. (The same controller served as the tactician for each of the participants.) Participants had as much time as they needed to develop their plan and were encouraged to make changes to their plan at any time.
After 10 minutes the scenario was again paused and participants and the tactician filled out a NASA-TLX workload questionnaire (Hart & Staveland, 1998) and a planning performance questionnaire. The scehario then resumed for another 10 minutes. At the end of the scenario, participants again assessed their workload and planning performance. The tactician also assessed his workload and the quality of the strategist's plan. After a short break the second scenario was started and proceeded analogously. Half the participants used FOPA first and half used the conventional flight progress strips to prepare their plan. At the end of the experiment, participants filled out a questionnaire that addressed the controllers subjective work experience with FOPA as well questions about its functionality.

RESULTS AND DISCUSSION
The use of FOPA was beneficial for planning performance when compared to using strips. A two-way repeated measures ANOVA (strips or FOPA vs. tactician or planner making the judgments) revealed that performance was better in the FOPA condition than in the strip condition, F(1,22) = 10.65 (all effects significant atp < 0.05 unless otherwise reported). No other effects were significant.
During the first ten minutes, planning performance using FOPA was only marginally superior to strips (p = 0.055). However, FOPA was far superior during the second half of the scenario (F(1,22) = 11.9). Performance during the first 10 minutes should largely reflect the quality of the initial plan, suggesting that the plans created with FOPA may be only slightly better than those created using strips (although the size of the advantage may be underestimated given the relative lack of familiarity). The robust advantage for FOPA in the second half of the scenario suggests that FOPA may be especially beneficial for updating and adapting strategic plans to changing traffic situations.
To investigate specific aspects of planning, the overall performance measure was broken down into its subscales: quality of initial plan, quality of mid-course revisions, planning effectiveness, and strategic awareness. During both halves of the scenario, participants showed significantly better planning effectiveness and better strategic awareness when using FOPA (see Table 1). Mid-course revisions were only better in the second half of the scenario when using FOPA, but this is sensible given that revisions became increasingly important as the scenario unfolded Table 1: Mean differences between FOPA and strip conditions on planning performance subscales: all scales range from I lo 7. Positive numbers indicate that FOPA resulted in higher ratings than when strips were used. Note: * indicatep < 0.05, ** indicalep < 0.01. Initial planning performance was measured only in phase I. FOPA yielded lower TLX workload ratings than the strip condition (F(l,22) = 10.87). There were also differences in workload ratings between tactician and planners with the tactician consistently rating his workload as higher. Importantly, however, the interaction between these two effects was not significant indicating that the workload of both the tactician and the strategist consistently decreased when FOPA was used. The workload decrease was visible during both the first and second 10 minutes of the scenario.
Workload and performance ratings were highly correlated. The tactician's judgment about his own workload and his evaluation of the participants plan correlated r = -0.89 in the interface condition and r = -0.9 in the strip condition (both correlations are significant). Thus, for both types of interface, the tactician's workload decreased as his rating of the quality of the strategist's plan increased. This result is not surprising; a good plan should entail a lower workload for its implementation while a bad plan should need more corrections and modifications.
Interestingly, the correlation between the participants' workload and their self-evaluated planning performance was not significant (in the interface condition r = -.48,p = 0.12, in the strip condition I = -.53, p = 0.08). The lower correlation between workload and self-evaluated planning performance suggests that a good plan does not necessarily involve more effort in its development. This is consistent with the findings of Gronlund et al. (1998) who found that successful plans were often sketchy and tilled'in as the scenario developed, while less successful plans were often too detailed. Developing a sketchy plan should involve lower workload than a detailed plan, consistent with a relatively low correlation between workload and plan quality.

3-68
Pmeeding.s of the IEA ZOOOIHFES 2000 Congrexs Participants had as much time as needed to develop their plans while the scenario was initially paused. Participants developed their plans faster using FOPA than using ships, despite their relative lack of familiarity with FOPA. The difference was rather large (323 s faster when using FOPA, t(l1) = 2.223), although considerable inter-individual fluctuations existed. One explanation for this advantage for FOPA was probably the result of the ability to create an initial aircraft sequence automatically with FOPA. All participants except one used this function to develoj, an initial sequence.
At the end of the experiment, participants filled out a questionnaire that assessed qualitative characteristics about air traffic planning when using FOPA. Half the participants mentioned the utility of the active linkage between the two types of information, planning tokens and radar targets, and all mentioned the usefulness of colors to tie the two together (Recall that if controllers categorized aircraft into the same queuing block, aircraft were grouped together by the same color on both radar screen and the flight organizer). Performing actions on the flight organizer screen and being able to observe the changes on the radar screen were also seen as highly useful. This active linkage between two different types of information constitutes one of the main enhancements to the traditional usage of flight progress strips. In the traditional air traffic environment, marking or x-categorizing strips has no effect on the aircraft representation on the radar screen.
Overall, FOPA proved to be a superior strategic planning aid in several ways. Not only was planning performance better, but workload was lower and plan development was faster. It is possible that these advantages arose because of the dynamic linkage between the two different types of air traffic information that are separated in the conventional air traffic environment. This reduces the time and cognitive effort needed to combine the two different representations of the same situation. Furthermore, the ability to automatically create a proposed sequence of air traffic is highly beneficial for the air traffic planner,