Reflecting on the Evidence: A Reply to Knight, McShane, et al. (2020)

Knight, McShane, et al. (2020) report three experiments on testosterone’s effect on the Cognitive Reflection Test (CRT). The experiments were designed and executed independently of each other and of our previous work (Nave, Nadler, Zava & Camerer, 2017). We thank Knight, McShane, et al. for conducting these experiments and summarizing their results, and we agree that one experiment is obviously not enough for establishing an empirical fact. The individual experiments and their meta-analytic summary are consistent with both the null hypothesis and Nave et al.’s conclusions (see Table S6 in Knight, McShane, et al.’s Supplemental Material), and there is evidence for variation in effects across experiments. In what follows, we reflect on design differences among the experiments and the collective evidence that their data contain