Randomization Tests and Multi-level Data in State Politics." State Politics and Policy Quarterly 10(2): forthcoming

Many hypotheses in state politics research are multi-level—they posit that variables observed at the state level affect individual behavior. When standard OLS is used to test hypotheses about state-level effects using individual-level observations, it yields overconfident standard errors and may lead researchers to falsely reject null hypotheses. Primo, Jacobsmeier, and Milyo (2007) explore this problem in their reanalysis of Wolfinger, Highton, and Mullin's (2005) data on the effects of voter registration laws on voter turnout. They advocate the use of clustered standard errors to solve the problem of overconfidence. In this paper, we offer an alternative approach. We reanalyze the same Wolfinger and Highton data using a randomization test. The randomization test proceeds in three steps. First, we randomly reassign registration laws from one state to another state, scrambling the key independent variables to ensure that there is no systematic relationship between laws and turnout. Second, we rerun the usual regression to estimate the coefficients on the law variables. We repeat this randomization technique many times in order to generate a distribution of simulated spurious effects. Finally, using this distribution, we can judge how likely it is that we observe the registration-law effect