An Evaluation of the Alignment Method for Detecting Measurement Non-invariance in Noncognitive Scales

In recent years a new methodology, the alignment method (Asparouhov & Muthén, 2014), has surfaced for estimating measurement models and detecting measurement noninvariance (i.e., DIF) across many groups. The purpose of the current study was to investigate the alignment method for use with non-cognitive scales across groups of students from different educational contexts (e.g., schools or programs). Asparouhov and Muthén (2014) have investigated the method with continuous and binary item scales, thus I extended previous research by using simulation techniques to evaluate the method with polytomous items, which are often used to measure noncognitive constructs. I also evaluated the new tests of noninvariance produced by the alignment method to a greater extent than has been seen in previous research. Results indicate that the alignment method adequately recovers parameter estimates under small and moderate amounts of noninvariance, with issues only arising in the more extreme conditions. The tests of noninvariance were found to be too conservative for most items, with a near zero Type I error rate. The testing procedure showed appropriate power in polytomous items that were less skewed in the mean structure, which suggests that the psychometric properties of individual items have a large effect on the performance of the procedure