Does systematic variation improve the reproducibility of animal experiments?

Reproducibility of results is a fundamental tenet of science. In this journal, Richter et al.1 tested whether systematic variation in experimental conditions (heterogenization) affects the reproducibility of results. Comparing this approach with the current standard of ensuring reproducibility through minimizing variation in experimental conditions (standardization), they concluded that heterogenization improved reproducibility1. However, in our view, they did not account for significant sources of dependency in their data, which resulted in an inflated type I error rate through pseudoreplication (defined as “the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent”2). We show that this leads to strong overconfidence in their analyses and that their hypothesis is unsupported