The coevolution of effort and replication, replicated and corrected

We replicated “The evolution of bad science” by Paul Smaldino and Richard McElreath (2016). The replication was successful with one exception. We find that selection acting on scientist’s propensity for replication frequency caused a brief period of exuberant replication not observed in the original paper due to a coding error. This difference does not, however, change the authors’ original conclusions. The two figures displayed here are recreations of Figure 5 – titled “The coevolution of effort and replication” – of the original study. One is based on the same coding error and the other is based on a corrected model. While effort, false positive rate (α), and false-discovery rate converge to the originally reported values when the simulated steps are extended beyond the original 1e6 time steps, the replication rate’s progress and convergence are different from the original. The results of this corrected model show the following pattern: Starting with a high effort, low effort replications are more attractive than conducting novel research (that is, employing this strategy received higher payoffs), which results in the replication rate reaching nearly 100% after ~730,000 steps. At this point the decline of effort has made low-effort novel research more attractive than low-effort replications (because publishing a novel positive result is associated with a higher payof than publishing a replication) and consequently the replication rate decreases again. With the decline of effort, alpha rises up to 0.67, comparable with the value reported in the study by Smaldino and McElreath (2016). Smaldino, P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3(9), 160384. https://doi.org/10.1098/rsos.160384