Type S error rate (i.e. published estimates with the wrong sign) as a function of the prevalence of true effects, statistical power and bias when different effect sizes are used for power calculations.

In (A), sample size is calculated to detect the mean of the true effect sizes above the minimum of interest (i.e. Cohen’s d > 0.5). This is the method used in the main simulations. As prevalence in the Single Normal model is increased by increasing the SD, the mean effect size above the minimum is larger as prevalence increases, leading simulations to be underpowered compared to other distributions at high prevalences. In (B), sample size is calculated to detect an effect equal to twice the minimum of interest (i.e. Cohen’s d = 1). In (C), it is calculated to detect an effect equal to the minimum of interest (i.e. Cohen’s d = 0.5). Each point corresponds to a simulation with 1,000 published findings. True effects are defined asthose above the minimum of interest. In each panel, graphs correspond to 20%, 50% and 80% power from left to right. Blue, red and green curves correspond to bias of 0, 0.2 and 0.6, respectively. (TIF)