On biases in assessing replicability, statistical consistency and publication bias

AbstractMethodology described by Francis in “Replication, Statistical Consistency and Publication Bias” is examined in the context of its application to the Topolinski and Sparenberg (2012) article. Several biases are discovered in this analysis, including selection biases in the reporting of p-values from the Topolinski and Sparenberg article, as well as in the criteria that were used in the selection of this article. General concerns regarding the replicability of scientific studies based on significance tests conducted at the 5% level of significance are also described