The power of statistical tests in meta-analysis

Calculations of the power of statistical tests are important in planning research studies (including meta-analyses) and in interpreting situations in which a result has not proven to be statistically significant. The authors describe procedures to com-pute statistical power of fixed- and random-effects tests of the mean effect size, tests for heterogeneity (or variation) of effect size parameters across studies, and tests for contrasts among effect sizes of different studies. Examples are given using 2 published meta-analyses. The examples illustrate that statistical power is not always high in meta-analysis. The use of quantitative methods to summarize the results of several empirical research studies, or meta-analysis, is now widespread in psychology, medicine, and the social sciences. Meta-analysis involves de-scribing the results of each study using a numerical index (an estimate of effect size such as a correlation coefficient, a standardized mean difference, or an odds ratio) and then combining these estimates across studies to obtain a summary. Although inference procedures for meta-analysis have been available for well over a decade, there is little work on the calculation of the power of statisti-cal tests in meta-analysis. However, power calcula-tions are always part of sound statistical planning (Co-hen, 1977). Moreover, power calculations are often a required component of research grant proposals in primary research, and the requirement of providing some estimate of statistical power is increasingly an issue in evaluating research synthesis projects as well. Although meta-analyses with large numbers of studies investigating even medium-sized effects may have quite powerful tests, meta-analyses of smaller num-bers of studies and meta-analyses in areas in which effects are expected to be small do not necessarily have very powerful statistical tests. The purpose of this article is to provide procedure