Evaluating factor structure and instrument stability through measurement invariance of the DECA, second edition

Abstract

In the process of instrument development, developers follow protocols and conduct analyses to ensure psychometric properties are met. During development, developers determine an assessment format that best aligns to their desired construct, including direct or indirect assessment. Within indirect assessment, participants hold a non-active role in the assessment process, where a separate informant supports the assessment process on behalf of the participant. Within this type of assessment, informant perception, being implicit and explicit biases, as well as informant memory implicate the results of the instrument; and therefore, the psychometric properties. This study sought to understand the factor structure and instrument stability via measurement invariance of the Devereux Early Childhood Assessment (DECA), second edition, within a sample of children enrolled in Head Start. Measurement invariance seeks to measure stability within populations to ensure the instrument is equivalent between groups. Further, achieving invariance within instruments means interpretations in observed change reflect actual change within the latent variable (Millsap, 2010). Exploratory and confirmatory factor analysis identified a four-factor structure. Most of the items within the total protective factors domain aligned to the factor structure described by the DECA. Six of those items cross-loaded; 9 of the 11 items within behavior concerns domain loaded to a factor within the total protective factors domain. Within measurement invariance, gender, race/ethnicity, dual language status, and time variables were examined. No invariance model met configural invariance. To further examine invariance by gender, items within the behavior concerns domain were excluded. Configural and metric invariance were met and partial invariance within strong and strict invariance were met. This notes the need to further examine the items included in the behavior concerns domain. Based on the results of this study, the argument that conducting reliability and validity analyses during instrument developer is insufficient. Developers should conduct more advanced analyses to ensure robustness and appropriateness within the population(s) for which the instrument has been devised. Finally, due to the nature of indirect assessment, these analyses are vitally important due to the potential error that may be imparted into an informant's judgement via perception and memory.