Reliability, validity, and equivalency of a computerized curriculum based measure for basic math skills

Abstract

This study examined the performance of third, fourth, and fifth graders on curriculum based measurements for math. The study sought to compare performance on two different computer formats to the traditional paper and pencil format. There were 133 initial participants and 80 participants after elimination for inaccuracy. Participants completed three each of the Paper/Pencil, Simulated Paper, and Flashcard probes. All probes were administered in a group format in a computer lab. A significant interaction was demonstrated, F(4,160) = 4.65, p = .001, such that participants performance on the assessments varied based on grade level and assessment format. There was a main effect of grade, F(2,80) = 33.17, p < .001, such that fifth graders obtained more digits correct than fourth graders and fourth graders obtained more digits correct than third graders. There was also a significant main effect for format, F(2,160) = 44.84, p <.001, such that there were significant differences between performance on the various formats. Due to multiple levels of each factor, it is necessary to examine simple effects and simple contrasts to determine the location of the significant differences. Significant differences were found for all grades between all modes with the exception of two comparisons. A comparison of paper to computer paper for third graders yielded a significance of p = .080. A comparison of computer simulated paper to flashcard yielded for fourth graders yielded a significance of p = .067. Both of these almost reach significance indicating that the performance on the three methods of assessment is often significantly different regardless of grade. All three formats demonstrated excellent reliability and could be used to assess progress. However, the computer formats that were developed and used for this study were not found to be equivalent to paper and pencil method.