Maximizing and Borrowing Information in Randomized Trials

To learn efficiently from randomized experiments, it is critical to understand how they may be designed and analyzed to best accumulate and interpret the statistical information that their data provide. To that end, this dissertation includes research on three important problems. In the first paper, we develop promising Bayesian uncertainty-directed (BUD) designs for faster and more informative dose-ranging clinical trials. The basic principle is to randomize new patients more often to doses that are expected to generate the most added information about the optimal dose, averaged over the posterior predictive distribution of their still unknown outcomes. This typically means assigning new patients to doses that are understudied relative to how strongly the data suggest they are optimal. We also use Bayesian model averaging of dose-response curves to robustly accelerate learning by letting each dose’s effectiveness partially inform those of nearby doses. This butts against a computational challenge that has made BUDs with nontrivial data models impractical, so we develop an efficient Sequential Monte Carlo strategy to enable this appealing approach to multi-arm trial design. In the second paper, we propose a new model for borrowing from historical controls in efficacy trials. This model is called SPx ("synthetic prior with covariates") and uses carefully posed Bayesian model averaging to balance between competing philosophies about how the historical and new data are related. In simulations and a case study we show how SPx quickly distinguishes between historical data that are helpful and historical data that are misleading, leading to a smaller control group in the new trial to the extent reasonable. In the third paper, we consider the often overlooked problem that in multi-site efficacy trials there are often substantive grounds to believe that the effectiveness of each site may be related to its size or randomization ratio. We call this phenomenon endogeneity of design. We re-evaluate treatment effect estimators commonly used in practice and derive asymptotic and finite-sample results as well as run extensive simulations to characterize their performance under this more realistic assumption. In a detailed case study of a landmark trial in education, we take a Bayesian viewpoint to evaluate the likely performance of the popular estimators in this specific setting. The implication is that endogeneity of design can significantly complicate analysis of multi-site trials, and existing methods are not well-equipped to handle this situation. For all three papers, code to reproduce the main analyses and simulations is included as supplementary files