229 Correlated Effects in Generalizability Studies

The analytical model typically used to perform generalizability analysis assumes that design effects are uncorrelated. Often, the assessment of behavioral data involves designs that employ multiple occa-sions or repeated trials (as in many observational and rating studies). In these cases, design effects may be serially correlated. The implications of serially correlated effects on the results of gen-eralizability analyses are discussed. Simulated data are provided that demonstrate the biases that serially correlated effects introduce into the results. Index terms: correlated effects, estimation of variance components, generalizability theory, observational studies, repeated trials, serial correlation