Simple Difference, and Residual Change Scores in Testing Two-Wave Data

This study compares the analysis of covariance (ANCOVA), difference score, and residual change score methods in testing the group effect for pretest–posttest data in terms of statistical power and Type 1 error rates using a Monte Carlo simulation. Previous research has mathematically shown the effect of stability of individual scores from pretest to posttest, reliability, and nonrandomization (i.e., pretest imbalance) on the performance of the ANCOVA, difference score, and residual change score methods. However, related power issues have not been adequately addressed. The authors examined the impact of stability of measurement over time, reliability of covariate and criterion, nonrandomization, sample size, and treatment effect size on statistical power of the three methods. Across conditions, ANCOVA and residual change score methods had similar power rates. When reliability was less than perfect, ANCOVA had more power than the difference score method when there was an increase from pretest to posttest and a positive baseline imbalance (i.e., treatment group had higher pretest scores than the control group), or when ther