'The Way in Which an Experiment is Conducted is Unbelievably Important': On the Experimentation Practices of Economists and Psychologists

To discuss experimental results without discussing how they came about makes sense when the results are robust to the way experiments are conducted. Experimental results, however, are – arguably more often than not – sensitive to numerous design and implementation characteristics such as the use of financial incentives, deception, and the way information is presented. To the extent that economists and psychologists have different experimental practices, this claim is of obvious practical and interpretative relevance. In light of the empirical results summarized below, it seems warranted to say that it does not make sense to report experimental results without reporting the design and implementation choices that were made.Duhem-Quine problem; experimental design; experimental implementation; financial incentives; deception