Meta-analyses, multivariate analyses, and coping with the play of chance.

We have some reservations about the proposal by Joseph Lau and colleagues (Jan 10, p 123)1 that, because of the diversity of patients' responses to treatment, meta-analyses must evolve to include response-surface models and other multivariate techniques. On what basis should patients or their carers decide which characteristics are sufficiently important potential treatment-effect modifiers that the necessary data should be collected and selected for inclusion in a multivariate response surface?There can never be any gilt-edged confidence that patients who participated in yesterday's research (and who will thus be represented in today's and tomorrow's meta-analyses) are comparable with today's patients. Even a wealth of data describing patient characteristics may fail completely to explain why some patients fare well and others badly.2 The modelling proposed by Lau and colleagues, when based on prior hypotheses, may sometimes provide a secure basis for inferring that treatment effect modifiers have been correctly identified; but the more that data from trials and meta-analyses are subdivided to look at finer and finer subgroups, the more one must expect false-negative and false-positive inferences. And, the more data on patient characteristics are sought, the more difficult it may be to compile a reliable overall estimate.3We are in full agreement with Lau and colleagues that further research is needed, but we suggest that such research should concentrate on exploring whether the goal of knowing how best to treat individual patients is best served by trying to interpret the inevitable uncertainties associated with a complex approach such as multivariate modelling, or the uncertainties represented by the range of the confidence intervals that surround every pooled-effect estimate. Indeed, perhaps the first challenge is to confront the excessive overreliance on point estimates and p values in both these circumstances