This study investigated differences in problems involving graphical representations of functions. Specifically, problems requiring students to answer questions about a given graph were compared with problems requiring students to construct a graph to meet specified conditions.

Numerical results suggested true systematic differences in problems types do exist. Information obtained from personal interviews revealed differing opinions regarding difficulty level of the problem types, which supported the numerical findings. Seventy-five percent of the students interviewed thought it was more difficult to construct a graph to meet specified conditions than it was to answer questions about a given graph. Although it might be expected that only higher ability students would think constructing a graph is easier than answering questions about a graph, in fact three of the students who held this opinion had the lowest final course grades. In addition, the numerical results indicated the students prepared differently for examinations than for quizzes. Information from the interviews revealed that this was due in part to the fact that the examination and quiz scores were weighted differently in the calculation of final grades. The interviews also revealed two basic strategies students used to construct graphs, in addition to several unexpected misconceptions held by the students.

A major result of this study is a warning against assuming comparability in assessment items involving graphical representations of functions. An additional result is a warning against the use of simple tests of differences in mean achievement to determine if items are similar and comparable. When the measurement situation is unclear, studies such as this generalizability study of what factors contribute to variance become especially important.

Subjects for the study were students enrolled in a first-semester calculus course during the 1998 fall semester at a medium-sized regional university. Thirty-two students, 13 females and 19 males, completed 24 problems involving graphical representations of functions. Half of these problems required students to answer questions about a given graph, and the other half required students to construct a graph meeting specified conditions. The students' written solutions to these problems were scored by two raters. Generalizability theory was used to study the differences between the two types of problems.