Placebo Tests for Synthetic Controls

The synthetic control (SC) method has been recently proposed as an alternative to estimate treatment effects in comparative case studies. An important feature of the SC method is the inferential procedures based on placebo studies, suggested in Abadie et al. (2010). In this paper, we evaluate the statistical properties of these inferential techniques. We first show that the graphical analysis with placebos can be misleading, as placebo runs with lower expected squared prediction errors would still be considered in the analysis. Then we show that a test based on the the post/pre-intervention mean squared prediction error, as suggested in Abadie et al. (2010), ameliorates this problem. However, we show that such test can still have some size distortions, even if we consider a case in which the test statistic has the same marginal distribution for all placebo runs. Finally, we show that the fact that the SC weights are estimated can lead to important additional size distortions