Selection bias in quasi-experiments

This thesis explores the role of selection bias in quasi-experiments, which are experiments where the treatment assignment is not random. The first chapter discusses several quasi-experiments from legal and sociological settings in which the detection of a treatment effect may be confounded with inherent pre-treatment differences in the treated and control groups. In the second chapter, we see that the literature on quasi-experimental designs contains many controversies, not all of them clearly and carefully developed. The third chapter gives a general, formal theory of selection bias in quasi-experiments. In the third chapter, we see that when statistics have a particular property, which I call removable selection bias, the treatment effect is estimable even in the presence of selection bias. The requirements for a statistic to exhibit removable selection bias are quite general and may be expressed in terms of concepts such as independence and conditional independence, without making explicit distributional assumptions. Traditional discussions of quasi-experiments have suggested that designs are free of selection bias only if there is no selection by maturation interaction, defined in terms of a linear model. I show that selection bias is poorly characterized by selection by maturation interaction. Selection bias may be removable in the presence of such an interaction, or in the absence of a linear model. While selection bias can be removed in a variety of situations, the result can be a statistic with higher variance. In the fourth chapter, numerous tables are given which demonstrate that some good designs yield statistics which remove selection bias with little or no increase in variance, while with other designs, the variance increases sharply. The fifth chapter explores the situation in which bias remains that cannot be removed. In this circumstance, we can gauge the sensitivity of statistics to such bias. In analyses of this sort, we see that factors related to selection but unrelated to the statistic can be ignored. In the sixth chapter, we discuss directions for future research