The effects of question context and response format on achievement in an aviation mechanic weight and balance assessment.

Abstract

This study investigated the effects of question context (word problems vs. abbreviated list) and response format (multiple-choice vs. constructed-response) on achievement in an aviation weight and balance quiz. An alternative question format was presented to simulate a more realistic problem situation. An experimental, computer-administered quiz was designed to measure the differences in test performance on the subject of weight and balance. Typical quiz items required the student to compute empty weight center of gravity. Other quiz items required knowledge of Federal Aviation Administration (FAA) rules regarding weight and balance measurements. A self-efficacy questionnaire was administered prior to the quiz. Distractors for the multiple-choice format were based on common misconceptions in interpreting FAA regulations.

A sample of 100 students from four aviation mechanic schools in Oklahoma and Texas participated in the experimental study. Scores for the multiple-choice questions were significantly higher than for the constructed-response questions. There was no difference in question context. Results suggested that (a) the multiple-choice format was easier than the constructed-response format, (b) students may not be learning procedures for solving real weight and balance problems, and (c) math computational skills were often weak. Analysis of student notes revealed no major differences in problem representation or solution between the question contexts and response formats.