Informing early childhood policy: An analysis of the sensitivity of a school readiness risk index to changes in indicator selection

Abstract

The importance of school readiness to both the future of an individual child and society as a whole has given rise to several state-specific indexes designed to measure county-level risk for starting school unprepared to learn. One such index is the Oklahoma School Readiness Risk Index (OK SRRI), comprised of indicators known to be associated with poor school readiness. Among several indicators considered for the index, the final set was determined on the basis of multivariate statistical methods. Selected indicators related to race/ethnicity, family structure and economics, and child maltreatment. No health-related indicators were included.

A limitation of indexes is that there are no agreed-upon best methods or established theoretical framework of measurement for their construction, which makes indexes subject to specification error. Many indexes are developed using reflective measurement models, which assume indicators reflect a unidimensional latent construct. Indexes, however, follow formative measurement models in which indicators define a multidimensional construct. The use of an erroneous measurement model has considerable implications for policy and resource allocation decisions.

This study examined the sensitivity of the OK SRRI to changes to the indicator set. An alternate index was created that reduced the number of racial/ethnic indicators and included those related to health, such as low birth weight. Indicator selection was guided by a theoretical framework based on transactional/ecological and cumulative risk models of child development, as well as assumptions of formative measurement models. Nearly one-third of Oklahoma's counties experienced considerable shifts in rank from the original to the alternate index. Most increases occurred for counties with high rates on at least one health indicator, while many decreases were among counties with high rates on multiple racial/ethnic-related indicators.

This study demonstrated that changes to the indicator set can change the meaning of a construct, which underscores the significance of the indicator selection process. Given the political nature of indexes, it is imperative that those with a stake in the outcomes be included in these processes. As most indexes related to social constructs are intended to inform policy and resource decision-making, this study has important implications for the field of index construction.