Statistical analyses that provide an effect size are to be preferred

Whenever possible a statistical analysis which provides a measure of the magnitude of the effect should be used. Unfortunately many commonly used statistical tests do not produce an estimate of the effect size. For example, χ^2 and Fisher’s exact tests are often used to analyse a binary outcome (e.g. surgical complications). These tests assess how likely the observed results were to have occurred if the was no difference between the two groups; however they do this without estimating the magnitude of an effect. Fortunately alternative statistical methods are readily available which allow for more complex analyses, regression analyses (e.g. logistic or Poisson regression) or if an unadjusted difference is preferred calculation of the difference in the proportions between groups along with a corresponding 95% confidence interval. In the era of big data, it is more important than ever not to rely upon statistical significance alone as very large datasets allow small (sometimes tiny) effects to be statistically detected which are nonetheless clinically irrelevant. Different effect sizes metrics exist and can be used according to the outcome and the analysis method used (e.g. linear regression provides the mean difference, whereas Poisson regression can be used to estimate a risk ratio). It is important that the estimate of effect size along with its uncertainty (e.g. 95% confidence interval) is reported. Doing so also allows for an assessment of how precisely the quantity of interest has been measured