A multidimensional approach to test item selection in a practical color vision test.

Abstract

Previous research evaluated the color vision requirements and perceptual factors related to interpretation of color-coded weather radar information and resulted in an 80-item developmental test. The objective was to use Item Response Theory (IRT) methods to select the most efficient set of those items without adversely affecting the reliability or item domain of the test. The resulting test was used to examine the effectiveness of using Located Latent Class Analysis (LLCA) to distinguish between individuals based on their capability of identifying colors on a CRT. The participants were 335 individuals with normal color vision and 200 with varying degrees of red-green types of color vision deficiency. BILOG-3 (with a 3-parameter IRT model specified) provided indices for discrimination, item difficulty, and guessing. The greater the discrimination value, the better the item discriminates high performers from low performers. Likewise, large difficulty values (in the positive direction) indicate more difficult items. Higher guessing values indicate a greater probability of guessing the correct choice. BILOG-3 also calculates an estimate of each individual's ability using the same scale as the item difficulty index. Using these indices, a composite scatterplot was constructed. With the item discrimination plotted on the Y-axis, and the item difficulty/ability estimate on the X-axis, boundary lines were drawn for the guessing parameter in addition to the minimum and maximum ability estimates for each anomaloscope diagnosis. Once the item's content was coded onto the scatterplot, items were evaluated on several dimensions to select the most efficient items. Maintaining the original pass/fail criterion, high consistency was found between the original 80 items and the 50-item test (K(172) = .95) as was the correlation between percent correct scores (r(172) = .99). The original inter-item reliability (alpha = .96) and item content were unaltered by shortening the test. LLCA latent trait scores, CTT percent correct scores, and IRT ability estimates were highly correlated; however, agreement between LLCA latent trait class assignments and pass/fail on the CWRT was moderate, K(172) = .62. The unique, multidimensional approach to item selection using the IRT 3-parameter estimates while taking item content and ability range information into consideration yielded 50 highly reliable items.