Pooling Correlation Matrices Corrected for Selection Bias: Implications for Meta-analysis

Selection effects systematically attenuate correlations and must be considered when performing meta-analyses. No research domain is immune to selection effects, evident whenever self-selection or attrition take place. In educational research, selection effects are unavoidable in studies of postsecondary admissions, placement testing, or teacher selection. While methods to correct for selection bias are well documented for univariate meta-analyses, they have gone unexamined in multivariate meta-analyses, which synthesize more than one correlation from each study (i.e., a correlation matrix). Multivariate meta-analyses of correlations provide opportunities to explore complex relationships and correcting for selection effects improves the summary effect estimates. I used Monte Carlo simulations to test two methods of correcting selection effects and evaluate a method for pooling the corrected matrices. First, I examined the performance of Thorndike\u27s corrections (for both explicit and incidental selection) and Lawley\u27s multivariate correction for selection on correlation matrices when explicit selection takes place on a single variable. Simulation conditions included a wide range of selection ratios, samples sizes, and population correlations. The results indicated that univariate and multivariate correction methods perform equivalently. I provide practical guidelines for choosing between the two methods. In a second Monte Carlo simulation, I examined the confidence interval coverage rates of a Robust Variance Estimation (RVE) procedure when it is used to pool correlation matrices corrected for selection effects under a random-effects model. The RVE procedure empirically estimates the standard errors of the corrected correlations and has the advantage of having no distributional assumptions. Simulation conditions included tau-squared ratio, within-study sample size, number of studies, and selection ratio. The results were mixed, with RVE performing well under higher selection ratios and larger unrestricted sample sizes. RVE performed consistently across values of tau-squared. I recommend applications of the results, especially for educational research, and opportunities for future research