Trade-off between Type I error and power: a requirement of robust technique for analysing ordinal outcomes

Background: A number of statistical models are available for analysing ordinal outcomes. Its application is increasing in all disciplines from clinical trials to various social experiments. A robust model is required as all the models for ordinal outcome variables have some shortcomings. Particularly, in case of detecting the small improvement or decoration of patient’s condition in clinical trials, the current methods have not shown significant promise. Sliding Dichotomy method is best possible model at present for analysing such ordinal outcomes. Purpose: This study showed the relationship among the three criteria, namely, sample size, type I error and power of the popular models that are frequently used for analysing ordinal outcomes. Phase III trial of traumatic brain injury has a five-point outcome scale which were analysed here using the binary regression model, proportional odds model, continuation ratio model and sliding dichotomy. Methodology: This study used CRASH dataset, a multicentre randomized controlled trial with sample size of 10,800. Varying sample sizes, number of covariates, treatment effects and band sizes of sliding dichotomy - in total using eight different models, type I error was quantified alongside statistical power. These shows how the models react to different scenarios in terms of maintaining false positive and power. Results & Conclusion: The results suggest, sliding dichotomy fails to control type I error rate showing high inconsistency. The proportional odds model and conventional dichotomy also showed inconsistency but false positive rate declined as sample size increased. But the power demonstrated by them was lower than the sliding dichotomy. This shows the importance of calculating type I error alongside finding significance of any test especially in case of public health researches. Also more sensitive and robust model is required that can maintain the 5% threshold of type I error and also achieve the best possible power within that range