CAN RELIANCE BE PLACED ON A SINGLE META-ANALY SIS?

Meta-analysis replaced statistical significance with effect size in the hope of resolving controversy concerning evaluation of treatment effects. Statistical significance measured reliability of the effect of treatment, not its efficacy. It was strongly influenced by the number of subjects investigated. Effect size as assessed originally, eliminated this influence but by standardizing the size of the treatment effect could distort it. Meta-analyses which combine the results of studies which employ different subject types, outcome measures, treatment aims, no-treatment rather than placebo controls or therapists with varying experience can be misleading. To ensure discussion of these variables meta-analyses should be used as an aid rather than a substitute for literature review. While meta-analyses produce contradictory findings, it seems unwise to rely on the conclusions of an individual analysis. Their consistent finding that placebo treatments obtain markedly higher effect sizes than no treatment hopefully will render the use of untreated control groups obsolete. Australian and New Zealand Journal of Psychiatry 1990; 24:405-415 Meta-analysis was introduced with the expectation it would provide an objective and non-controversial evaluation of the results of a series of studies. Litera-ture review by reasoned discussion, the method pre-viously used to evaluate such studies commonly resulted in differing conclusions being reached by different reviewers. These differing conclusions fre-quently resulted from differences in the importance attached by the different reviewers to whether or not the studies produced statistically significant findings. The concept of statistical significance was introduced to allow a decision to be made as to whether or not an effect was reliable, that is, replicable. If the probability that it had occurred by chance was less than a conven-tionally accepted figure (usually equal to or less than five percent), it was agreed that the probability coul