A generalization of null hypothesis significance testing with applications to replication failures, molecular evolution models, and bounded parameter spaces

Null hypothesis significance testing is generalized by controlling the Type I error rate conditional on the existence of a non-empty confidence interval. The control of that conditional error rate results in corrected p-values called c-values. A further generalization from point null hypotheses to composite hypotheses generates C-values. The framework has implications for the following areas of application. First, for bounded parameter spaces, C-values of unspecified catch-all hypotheses provide conditions under which the entire statistical model would be rejected. Second, the C-value of a point estimate or confidence interval from a previous study determines whether the conclusion of the study is replicated, discredited, or neither replicated nor discredited by a new study. Third, c-values of a finite number of hypotheses, theories, or other models facilitate both incorporating previous information into frequentist hypothesis testing and the comparison of scientific models such as those of molecular evolution. In all cases, the corrections of p-values are simple enough to be performed on a handheld device