Summary

The primary scientific goal of a randomized clinical trial of two treatments, A and B, is to compare their effects on the most important therapeutic outcome in the medical setting under study. Typically, this comparison is formulated in terms of the probabilities or hazards of the outcome with A and B, possibly transformed or covariate-adjusted. When the data arise from separate trials of A and B, however, standard statistical estimators of the A-versus-B treatment effect may be severely biased and substantively misleading due to confounding trial effects. Potential sources of systematic bias include supportive care, physician practices, covariate imbalances, and institutional effects. Although this problem is well known in the statistical community, and is the motivation for randomizing patients between treatments, it is common practice throughout the medical literature to base statistical comparisons on data from single-arm trials while ignoring trial effects. Because numerous medical decisions are based on such flawed analyses, this constitutes a serious problem, with consequences that go far beyond the scientific community. This chapter will provide illustrations of between-trial effects, and present some simple Bayesian methods for evaluating what the possible treatment effects may be in such settings