Reliability Generalization: Exploring Score Reliability Variance with Ryff’s Scale of Psychological Well-Being

The purpose of this study was to conduct a Reliability Generalization (RG; Vacha-Haase, 1998) for Ryff’s Scale of Psychological Well-Being (PWB; Ryff, 1989) to characterize the average score reliability, the variability of the score reliability, and explore possible sample and test characteristics that influenced score reliability across studies. Studies were included in the current investigation if they had been published in a peer-reviewed journal, used one or more subscales of the Ryff’s PWB, estimated coefficient alpha value(s) for the PWB subscale(s) used, and were written in English. Out of the 924 articles generated by the search strategy, a total of 264 articles were included in the final sample for meta-analysis. The average coefficient alpha for the composite PWB scale was 0.858, with mean coefficient alphas ranging from 0.722 for the Autonomy subscale to 0.801 for the Self-Acceptance subscale. Statistically significant heterogeneity was present across all mean coefficient alphas (p < .05), with the heterogeneity index above 95% for both composite and subscale alphas. Consequently, select sample and test characteristics of the primary studies were explored as possible moderator variables on coefficient alpha estimates, with significant differences in score reliability estimates across select demographic and test characteristics. Test length accounted for the majority of variance among alpha coefficients with R2 values ranging from 40% on the Environmental Mastery subscale to 71% on the Self-Acceptance subscales across the primary studies. In light of the current findings, implications for researchers using Ryff’s PWB including informed score reliability reporting practices are discussed