Invited Opinion Statistical Significance and Statistical Power in Hypothesis Testing

Summary: Experimental design requires estimation of the sample size required to produce a meaningful conclusion. Often, experimental results are performed with sample sizes which are inappropriate to adequately support the conclu-sions made. In this paper, two factors which are involved in sample size esti-mation are detailed-namely type I (a) and type I1 (p) error. Type I error can be considered a “false positive ” result while type I1 error can be considered a “false negative ” result. Obviously, both types of error should be avoided. The choice of values for a and p is based on an investigator’s understanding of the experimental system, not on arbitrary statistical rules. Examples relating to the choice of a and p are presented, along with a series of suggestions for use in experimental design. Key Words: Experimental Design-Type I error-Type I1 error-Sample size-Statistics. Statistical methods are often used in scientific settings to determine whether experimental results are true in general, i.e., whether results obtained from an experimental sample can be generalized to the population. We are familiar with the phrase “the results were significant (p < 0.05)... ” in the scientific literature. For some experimentalists, if the results are not “significant, ” they are not con-sidered worth reporting. To others, statistical anal-ysis seems to be a burdensome numbers game that is not necessarily related to scientific quality. In fact, it is incorrect either to “live and die ” by the p value or to avoid the use of statistical analysis be-cause of potential abuse. With the advent of modern statistical software packages (l), p values are ob