The replication of non-inferiority and equivalence studies

Replication studies are increasingly conducted to assess credibility of scientific findings. Most of these replication attempts target studies with a superiority design, but there is a lack of methodology regarding the analysis of replication studies with alternative types of designs. In order to fill this gap, we adapt three approaches used for superiority settings to non-inferiority and equivalence designs: the two-trials rule, the sceptical p-value approach and the meta-analysis criterion. While the adaptation to the non-inferiority design is relatively straightforward, the equivalence design requires more effort. We propose to use the 'two one-sided test' (TOST) procedure and tailor it to the replication setting. In addition, we formulate the different criteria in terms of a 'success interval' for the relative effect size (replication to original). The properties of the different approaches are studied in detail. In particular, the two-trials rule does not penalize replication effect estimates very close to the margin for large sample sizes. The meta-analysis criterion is convenient as it combines the two estimates into one, but can lead to replication success in odd situations. Finally, the sceptical p-value approach seems well-suited as it penalizes a too large increase of the replication effect estimate as compared to the original one, while taking into account the direction of the latter in the calculation of the success interval.Comment: 22 pages, 5 figure