Regression without Regrets: A modular approach to linear models in (quasi)-experiments.

The design of a randomized study guarantees not only clear and “in-terpretable comparisons”(Kinder and Palfrey, 1993, page 7) but valid statistical tests even in the absence of large samples or known data generating processes for outcomes (Fisher, 1935, Chap 2). Yet, while design alone yields valid tests the tests could lack power: a valid but wide confidence interval may be more useful than a misleadingly narrow confidence interval, but still shed little light on the theory motivating the study. After a brief demonstration of Fisher’s statistical framework (to fix ideas about the validity of tests and to distinguish it from frame-works which estimate average treatment effects), we show a method by which a researcher may use substantive background knowledge about outcomes in order to increase the power of her statistical tests. Com-bining substance and design in this particular way enables valid and powerful tests. We build on methods developed by Hansen and Bowers (2009) and with a long history in survey sampling and combine them with techniques of machine learning/data mining/statistical learnin