An evaluation of intensive care severity of illness scoring models

Objectives: To evaluate the accuracy of the four main Intensive Care severity of illness scoring models using a large Scottish database, and to investigate different strategies for improving their accuracy in a Scottish setting.Method: Twenty two out of 25 general adult Intensive Care Units in Scotland collected data for two and half years to allow calculation of Acute Physiology and Chronic Health Evaluation (APACHE) version II and III, Simplified Acute Physiology Score (SAPS) version II, Mortality Probability Model (MPM) version II (calculated on admission and at 24 hours). The models' Goodness of Fit (discrimination and calibration) and performances in subgroups (Uniformity of Fit) were evaluated using Receiver Operating Characteristic Curves, Hosmer-Lemeshow Goodness of Fit test, Chi Squared test and Confidence Intervals. Three of the Models (APACHE II, SAPS II, and MPM II) were customised with Scottish data using logistic regression techniques.Results: All models had good discrimination but poor calibration. However, the SAPS II and APACHE II models appeared to have better calibration than other models.All models, except the new APACHE II model, showed significant differences in important subgroups.Conclusions: Questions remain about the accuracy of these models even after customisation. Further research is needed to investigate variations in Intensive Care Units and the relationship to clinical effectiveness. However, where case mix adjustment is needed the new customised models remain the most accurate means of doing this in Scottish dat