Confidence Intervals for the Expected P-value

The p-value is widely used in many application fields. In common practice, a scientific finding is deemed statistically significant if its resultant p-value is less than a pre-specified significance level, for example α = 0.05, albeit many statistically significant results are not reproducible in new studies. Mixed reasons including misuses, abuses, misunderstanding and misinterpretation arouse intensive debates and conservatives around the p-value from time to time over the years. Yet no reasonable solutions have been proposed. In this research, we make efforts to close the gap by advocating the use of confidence level for the expected p-value p0. This allows us to perform a second-level testing problem (H0\u27: p0 �0.05 vs.Ha\u27: p0 \u3c 0.05.) for each hypothesis testing problem (H0 vs. Ha). Bootstrap-based confidence intervals are put forward. In particular, we investigate an infinitesimal jackknife (IJ) approach that possibly reduces variance in certain scenarios. The proposed method is empirically assessed and illustrated via both simulation studies and analyses of real data sets