Estimation of the predictive power of the model in mixed-effects meta-regression: A simulation study

Several methods are available to estimate the total and residual amount of heterogeneity in meta-analysis, leading to different alternatives when estimating the predictive power in mixed-effects meta-regression models using the formula proposed by Raudenbush (1994, 2009). In this paper, a simulation study was conducted to compare the performance of seven estimators of these parameters under various realistic scenarios in psychology and related fields. Our results suggest that the number of studies (k) exerts the most important influence on the accuracy of the results, and that precise estimates of the heterogeneity variances and the model predictive power can only be expected with at least 20 and 40 studies, respectively. Increases in the average within-study sample size (N) also improved the results for all estimators. Some differences among the accuracy of the estimators were observed, especially under adverse (small k and N) conditions, while the results for the different methods tended to convergence for more optimal scenarios.status: publishe