Variations in online survey designs

Abstract

The purpose of the current study was to extend the literature on the use of survey design elements in the construction of online surveys. More specifically, this study aimed to assess the effects of online survey design characteristics on participants' responses to online questionnaires presented in different visual survey designs.

The 216 participants in this study were Oklahoma State University (OSU) students, enrolled at the OSU-Stillwater campus during the Fall 2013 semester. Sixteen individual survey testing conditions were created in which the survey content was identical but the visual design characteristics varied. These conditions were designed with the purpose of revealing statistically significant differences between survey conditions.

The four independent variables used in the study were selected based on their relatively prevalent use in online survey design. Participants responded to questions on three instruments (i.e., knowledge test, attitudinal questionnaire, and computer self-efficacy scale). Response rate and completion time were also measured. Regardless of the visual presentation, all participants viewed the same questionnaire items (i.e., content).

A series of four 2x2x2x2 completely randomized factorial (CRF) analysis of variances (ANOVAs) were performed in order to determine the effects of the independent variables on the dependent measures. Qualitative analyses were also performed and data involved identifying and analyzing participants' text responses to an invitation to provide feedback regarding their experience during the survey.

This study implored a mixed mode strategy in which different forms (N=16) of the same online survey were administered to different groups. In general, participants revealed a strong dislike for the survey colors selected for this study. Taken together, the results of this study provide considerable evidence for the influence of color in online survey design. While the present study explored the effects of color on participants responses to questionnaire items presented in an online format, the unanticipated effects of the participants' physical environments necessitates future investigation. Future studies should bear in mind that surveys should be designed to be aesthetically pleasing and incorporate color in a way that makes the survey interesting while producing salient, quality data.