Incorporating Latent Variable Outcomes in Value-Added Assessment: An Evaluation of Univariate and Multivariate Measurement Model Structures

The purpose of this study was to systematically evaluate the degree to which univariate and multivariate measurement model structures were related to value-added model parameters, teacher estimates, and rankings. Most value-added assessment methods use a single test score to estimate teacher effects, but reliance on a single test score assumes that scores are an error-free approximation of the latent construct of academic achievement. The unique contribution of this study was the systematic evaluation of both univariate and multivariate measurement model structures in a particular value-added model to examine the utility of incorporating latent variable approaches within a traditional value-added framework. The proposed study was a 4x2x2x2x2x2 mixed factorial design. Value-added models varied by the degree to which teacher estimates were allowed to persist over time (complete, partial, half-life, and zero) and within each persistence assumption measurement model structures varied by two levels of measurement assumptions (parallel and non-parallel indicators), two levels of variable combinations (univariate and multivariate), two levels of scaling [Item Response Theory (IRT) scale scores and normal curve equivalent (NCE) scores], two levels of longitudinal invariance (non-invariant and invariant), and two levels of dependent variables (static achievement scores and change scores). Model parameters and teacher rankings were evaluated to assess the degree to which results were consistent across experimental combinations. The results showed that multivariate models, particularly the ones from the static achievement score conditions, were more stable across experimental conditions than the univariate. Although there was one type of univariate model that demonstrated strong rank-order correlations, the univariate models were less stable as a whole. Additionally, the multivariate models produced fewer outliers than the univariate models, indicating that they were less susceptible to bias across the experimental conditions examined in this study. The current study used empirical data to evaluate the consistency of model parameters and rankings, but future research will need to evaluate the degree to which univariate and multivariate models recover known population parameters. Advisor: James A. Bovair