at the Annual Meetings of the Canadian Political Science Association

Survey experiments help establish causality, but scholars do not know how closely the treatments mimic natural phenomena. This study compares survey experiments and a natural experiment on the same topic. In two survey experiments providing information about Medicare, we observe double-digit learning effects. In contrast, most respondents in our contemporaneous natural experiment show little evidence of learning. Consistent with our expectations, the only people who showed comparable levels of learning to respondents in our survey experiment were individuals exposed to Medicare facts in their media source of choice as well as people who were uncertain about the facts from the very beginning. Our conclusion is that survey experiments, at least on this topic, generate effects that are only observed among parts of the population who are likely to be exposed to treatment messages or predisposed to accept them