1 Simpson’s Paradox and the implications for medical trials

This paper describes Simpson’s paradox, and explains its serious implications for randomised control trials. In particular, we show that for any number of variables we can simulate the result of a controlled trial which uniformly points to one conclusion (such as ‘drug is effective’) for every possible combination of the variable states, but when a previously unobserved confounding variable is included every possible combination of the variables state points to the opposite conclusion (‘drug is not effective’). In other words no matter how many variables are considered, and no matter how ‘conclusive ’ the result, one cannot conclude the result is truly ‘valid ’ since there is theoretically an unobserved confounding variable that could completely reverse the result. 1. Introducing Simpson’s paradox “Simpson’s paradox ” is a commonly occurring statistical phenomenon first formally recorded in 1951 [6] and has been addressed in [7] and [5]. For reasons that we will make clear in this paper, it is not really a paradox even though it can often appear to look like one. A classic and simple example of Simpson’s paradox is the scenario in which Jane’s scores are worse than Fred’s every year but she still comes with better overall average. How is this possible? Fred and Jane study on the same course spread over two years. To complete the course they have to complete 10 modules. At the end, their average annual results are as shown in Table 1