Individual Differences in Criterion Task Set Performance1

As part of a larger standardization study of the U.S. Air Force Criterion Task Set (CTS), a number of individual difference variables were assessed to evaluate their relationship to CTS performance. The selection of these individual difference variables was based on their known or hypothesized relationship to performance or perceptual abilities. This paper reports some of the preliminary findings with regard to these individual difference variables. Of the variables measured, Stimulus Screening, Thrill and Adventure Seeking, Neuroticism, Type–A Behavior, and General Intelligence showed the most promising relationships to CTS performance variables.


INTRODUCTION
There is a growing awareness that human capabilities have become the limiting factor in many operational systems. Designers a r e now faced with situations whertt operational systems a r e limited by the inforniation processing, decision making, memory, mental alertness, or physical capabilities of the operator-.
In response to the growing awareness of these problems the U.S. Air Force has dedicated considerable effort to the exploration of workload assessment metrics. One major result of these efforts has been thc development of the Criterion Task Set (CTS), a set of standardized loading tasks that can be used both to evaluate potential workload measures and to provide graded workload levels of it number of basic performance tasks for a wide range of human factors research (Shingledecker, Acton, & Crabtree, 1983).
The CTS Version 1.0 is composed of nine tasks.
Eight of the tasks have three graded workload levels. These tasks include: Memory Search, Continuous Recall, Linguistic Processing, Probability Monitoring, Grammatical Reasoning, Mathematical Processing, Unstable Tracking, and Spatial Processing. The ninth task is a single-level lnterval Production (tapping) task.
An important characteristic of the CTS is that it is one of the first task batteries to be developed within the framework of modern information processing theory. Thus, the CTS 1.  Schlegel, 1985).

This research was sponsored by the Workload and Ergonomics
The resulting data baso from this study includes riunierous perforrriancc: and subjective workload tncasures for each of the CTS tasks collected undcr slandiir-d laboralory conditions, as well as, during exposure to common environmental stressors (time press, noise, sleep loss). A descript.ion of the experimental methodology and the CTS performance results of this study are reported elsewhere in those Proceedings of the Human Factors Society 30th Annual Meeting (See Schlegel, Gillilond, & Schlegel, 1986).
A s a part of this large-scale study, subjects w e r e assesscd on a number of individual difference dimensions that have known relationships to performance efficiency and/or are hypothesized to be biologically or perceptually based. There a r e several individual difference variables that are known to relate to the manner in which a person processes information or to the processes directly related to performance capability (e.g., arousal state). Currently, there a r e scales that assess the arousal dimension (Eysenck & Hysenck, 1968), as well as related issues such as the degree of sensation seeking (Zuckerman, Kolin, Price, & Zoob, 1964). There are also scales which assess perceptual processing ability (Mehrabian, 1977;Sarason, 1972). It seemed prudent in such a foundational investigation of human performance that psychometric measures such as those described above should be included.
The purpose of this paper is to report on a preliminary analysis of the relationships between the individual difference variables and the CTS perforrrianct: variiibles derived from this large-scale standardization and validation effort,.

METHOD
As notcd previously, a detailed description of the methodology used in the large-scale standardization and validation study is reported elsewhere in these Proceedings. Briefly, the testing protocol consisted of regularly scheduled two-hour testing sessions conducted once per day, for ten days, over a two-week cycle.
Multiple workstations allowed for the simultaneous testing of four subjects during each two-hour session. Tes t ing sessions were scheduled beginning a t 8:OO a.m., 1O:OO a.m., 1:OO p.m., and 3:OO p.m.

Subjects
Twenty-five men (ages 19 to 32 years; mean = 23.6 years) and twenty-five women (ages 18 to 43 years; mean = 23.0 years) served as volunteer wibjects in this project. All subjects were recruited through posted annouricements and were paid for their participation in the study. All subjects reported 20/20 actual or corrected vision, no history of hearing impairment, and no current use of medication.

Apparatus
The CTS Version 1.0 tasks were presented on Commodore 64 microprocessor systems with dual floppy disks and a color CRT monitor.
Additional software w a s developed during this project to automate the presentation of CTS tasks, reduce data, and automatically label and store raw data and summary statistics files.
At each workstation the subject was provided three response controllers designed for the CTS battery a t the Workload and Ergonomics Branch at Wright-Patterson Air Force Base.
These consisted of a tapping button controller box for the Interval Production task, a turn-pot controller box for the Unstable Tracking task, and a four-button keypad for the remaining central processing and input/perceptual tasks. Gcneritlizsd Arousal. The Eysenck Personality Trivf:ntory w a s used to assess gcncralized anxiety (Eysenck & Eysenck, 1968). This dimension is believed to be directly related to brairisterri reticular formation activity which is subsequently reflected in different levels of cortical arousal.

Psychometric Tests
Introverts are hypothesized to be higher in arousal than extraverts. This arousal difference often leads to one group or the other being at a performance advantage depending on the environmental or task circumstances.
Reviews of both performance arid psychophysiological literature generally support this theory. This inventory also yields the Neuroticism scale score.
Stimnlus S c r e c n i n g . Also related to the orionting rcflex, as well as arousability, is the dimension of stimulus screening (Mehrabian, 1977).
Si.imulus screening refers to a bioloKic:i~lly-basc:d, perception-related dimension that reflects one's ability to screen relevant from irrelevant stimuli during information processing.
Sensation S e e k i n g . Developed from early sensory dcprival.iori and optiniiil level of arousal research, the sensation seeking scale (Zuckerman, 1979;Form V ) assesses the degree to which people actively seek sensory stimuli to increase their stimulation lcvel. This dimension has been related to orienting reflex diffcx-ences (Zuckerman, 1972), and to regulators of neurotransmitters (Murphy, Belmaker, Ruchsbaum, Martin, Ciaranello, & Wyatt, 1977). This scale also provides a subscale assessing the degree to which an individual actively seeks Thrill arid Adventure.

Clinical ( T r a i t ) Anxiety.
Clinical anxiety in n more general sense is simply termed anxiety, as opposed to more specialized forms such as test anxiety.
Anxiety is known to disrupt motor porformance arid cognition. Anxiety is usually viewed as being either of a transient "state" form, often due to situational factors, or i i more pervasive protracted "trait" form. Both trait and state anxiety w e r e assessed with the State-Trait Anxiety Inventory (Spielberger,Gorsuch,& Lushene,' I 970).

I m p u l s i v e n e s s .
Impulsiveness has been shown to be related to physiological processes, specifically arousal mechanisms. Impulsiveness w a s measured with the Barratt Impulsiveness Scale (Barratt, 1965) which also provides subscales of Non-Planning and Motor Impulsiveness.
Test A n x i e t y . Test anxiety is a form of anxiety associated with demand for performance.
One scale of test anxiety (Sarason, 1972) has shown negative correlations with performance efficiency, especially on vigilance and selective attention tasks.
Intelligence. While the theoretical nature of general intelligence remains controversial, this dimension has been shown to be a mediating factor in the performance of many tasks.
The Wechsler Adult Intelligence Scale-Revised (WAIS-R) was administered to all subjects in the study. Type A Behavior. Type A Behavior refers to a specific coping style which has been linked to coronary prone behavior. This dimension is interesting for two reasons. First, it shows an apparent relationship to physiological processes, e.g., cardiovascular responses. Second, it appears to be related to highly organized, stressful, competitive, and overscheduled approaches to problem solving. This dimension w a s measured with the Jenkins Activity Survey (Jenkins, Zyanski, & Rosenman, 1979).

Procedure
Subjects w e r e seated in their individual workstations facing the elevated CRT display. The controller boxes were placed on a table in front of the subjects. The workstations were separated by acoustic panels to reduce noise and subject interaction.
During the first two-hour session each subject w a s oriented to the project, given an introduction to each of the CTS tasks, completed a SWAT Sort (described below), and completed the battery of psychometric tests. In the second through sixth sessions, subjects were given the five training trials on the entire CTS battery.
The seventh and ninth sessions provided baseline experimental data collected under standard laboratory conditions, Le., the s a m e conditions imposed during training. Data from the eighth and tenth sessions were collected during exposure to common environmental stressors.
Only performance data from the seventh session (the first baseline session) was included in the analyses reported in this paper.
During each session subjects were presented three-minute trials of each of the 25 CTS t a s k s (three workload levels of eight tasks, plus the one level Interval Production task). Following each trial w a s a brief, 1-1.5 minute rest period during which data was stored on the diskette, the next task was prepared for presentation, and subjects completed subjective workload assessments.
The total testing t i m e per session w a s approximately one hour and forty-five minutes. Throughout the study, subjects were asked to provide subjective assessments of the workload presented by the various CTS tasks by the use of the Subjective Workload Assessment Technique (SWAT). The SWAT Scale (Reid, 1982;Reid, Eggemeier, & Nygren, 1982) is a psychometric instrument for measuring subjective ratings on three major dimensions of workload: Time Pressure, Mental Effort, and Psychological Stress.
The unique aspect of SWAT is that it not only provides a means for obtaining an individual subject's workload ratings, but is aIso a method for establishing cross-subject comparability. Table   1 presents the preliminary correlational analyses of the individual difference variables with a selected group of CTS task performance variables. Only correlations significant at or below the p =.05 level of significance are listed in Table 1.

RESULTS
It i s important to note that these general results are quite preliminary and based on a relatively low number of subjects (N=50). Nonetheless, these data do provide some encouraging information and yield some interesting trends which will be analyzed and reported more fully when collection of the full standardization arid validation data set is completed.
The individual difference variables included in Table 1  Two trends emerged in the correlational matrix which are important.
First, there is a group of individual d iffererice variables that seem to have significant relationships with multiple CTS tasks.
Second, five of the CTS tasks seem to have significant relationships with various individual difference measures.
T h e individual difference variables that seemed to show the strongest relationship to a number of CTS porforrnancc variables were: Stimulus Scxeening, Neuroticism, Thrill and Adventure Scxking, Intelligence, and Type A Behavior.
Stimulus Screening w a s negatively related to the mean response t i m e of Memory Search, Continuous Rocnll, Linguistic Processing, Grammatical Reasoning, Mathematical Procxssing, Unstable Trac:king, and Spatial Proce s sing.
These results suggest that the higher a person scores on this scale, that is, the more they show the capability to screen out irrelevant stimuli during information processing, the greater the likelihood that they will perform Inore rapidly on a number of CTS tasks that draw upon a fairly wide range of abilities. However, in only one case was accuracy of responding related to Stimulus Screening. I t appears that the higher one scores on Stimulus Screening more likely one is to be correct in the Grrirrimatic:al Reasoning Task. Screeners were also less likely to make Edge Violations in the: Tinstablo Tracking task. It should also be rioted that these relationships did not hold for all workload levels. *Correlations listed in the table were those found to be significant at least at the p=0.05 level.
Neuroticism was also significantly related to several CTS performance variables. Neuroticism, it should be noted, is best viewed as a dimension of emotional stability rather than an index of neurotic behavior in the strict clinical sense. Those individuals who scored higher in Neuroticism (Leo, those more emotionally unstable) were also more likely to respond more slowly in Memory Search, Linguistic Processing, and Math Processing. They were also more likely to commit Edge Violations in Unstable Tracking and respond incorrectly more often in Grarnmutical Reasoning.
Again, these relationships d-id not hold for. all workload levels.
Because Stimulus Scr*eeni.ng and Neuroticism w e r e found to be correlated I' = -.67 ( p <.0001), it should riot be surprising to find them having apposite pattcrns of correlations with CTS performanc:t: variablos.
Thrill and Adventure Seeking, a subscale of the Sensution Seeking Scale, was also related to several C?'S perforrnilnce vtiriables.
I t appears that thc: ~i i o r o one s c e k s thrill and adventure, the more likely one is to respond quickly to the Linguistic Processing arid Spatial Pr0cessin.g tasks. These individuals a r e also more likely to be correct in Memory Search and Spatial Processing.
Again, these results do not hold at all workload levels for d l tasks.
Intolligorm: was cit:ariy I,ht: variable rrlost related to C'I'S task performance. The more int.elligcnt onc is, as measured by the WAIS, the more I.ikely one is to respond rapidly to the Memory Search, Continuous Recall, Linguistic: Procxssing, Matheniiitical Processing, Spatial Processing, and Grammatical Reasoning tasks. More intelligent subjects were also more likely to be correct on the Memory Search, Continuous Recall, Linguistic Processing, and Grammatical Reasoning tasks, but not on the Mathematical and Spatial Processing tasks. In rnost cases, these findings generalized to more than one workload level and in some cases to all workbad levels. Also, it is interesting to note that motor output tasks (Interval Production and Unstable Tracking) seem to be fairly independent of intelligence.
There were other significant relationships of interest. However these relationships showed less consistency and therefore may be less reliable.
For example, Type A behavior seemed to be associated with an unusual cornbination of speed in Mathematical Processing and Spatial Processing.
Also, E d g e Violations in Unstable Tracking seemed to be related to Non-planning Impulsiveness (rather than Motor Impulsiveness, as one might predict), and Neuroticism.
It is also interesting to note that some CTS tasks correlated more frequently with the individual difference measures than others. Linguistic Processing and Grammatical Reasoning had the highest number and most complete pattern of correlations between both speed and accuracy measures and intelligence score-undoubtedly due to the emphasis on verbal fluency and facility in these activities. However, Linguistic Processing w a s related to several other individual difference dimensions while Grammatical Reasoning was significantly related to only a few others. It is also interesting to note that Interval Production appears to be unrelated to any of the individual differences dimensions--despite literature linking some of these variables to differences in rhythmic behavior o r task performance (e.g., Extraversion, Impulsiveness).
I t is clear that enough relationships of interest exist to warrant more extensive analyses of these data when a larger sample size is obtained.
The larger sample size will allow more power for the analyses, as well as provide the opportunity to perform more sophisticated multivariate analyses. In addition, the completed standardization data set will allow analyses of the relationship between individual difference variables and CTS performance variables across environmental stress conditions, i.e. , cross-situational paradigm research. Finally, the completed data set will also provide the opportunity to analyze the rc:lationship between individual difference variables and subjective workload assessments.