P-values

A p-value is the probability of observing a result as extreme or more extreme than that observed given that the null hypothesis (H0) is true. To be able to calculate a p-value, one thus requires at least three things: a null hypothesis, a probability model, and an agreed ordering of the possible outcomes, so that those that are more extreme than those actually observed may be identified. This ordering is generally conveniently arranged in terms of a “test statistic” T so that higher or lower values of T, as the case may be, or higher absolute values (as perhaps in two-sided p-values) identify more extreme cases. In fact, the requirement to order the outcomes also generally entails a fourth requirement—that all possible relevant outcomes have been defined or restricted. These relevant outcomes form the so-called sample space. Usually, because p-values are a frequentist concept, the sample space is defined in terms of the possible outcomes that could occur from an infinite repetition of the experiment (e.g., a clinical trial)