External validation of the ROSC after cardiac arrest (RACA) score in a physician staffed emergency medical service system

Background: The return of spontaneous circulation (ROSC) after cardiac arrest (RACA) score may have implications as a quality indicator for the emergency medical services (EMS) system. We aimed to validate this score externally in a physician staffed urban EMS system. Methods: We conducted a retrospective cohort study. Data on resuscitation attempts from the Helsinki EMS cardiac arrest registry from 1.1.2008 to 31.12.2010 were collected and analyzed. For each attempted resuscitation the RACA score variables were collected and the score calculated. The endpoint was ROSC defined as palpable pulse over 30s. Calibration was assessed by comparing predicted and observed ROSC rates in the whole sample, separately for shockable and non-shockable rhythm, and separately for resuscitations lead by a specialist, registrar or medical supervisor (i.e., senior paramedic). Data are presented as medians and interquartile ranges. Statistical testing included chi-square test, the Mann-Whitney U test, Hosmer-Lemeshow goodness of fit test and calculation of 95% confidence intervals (CI) for proportions. Results: A total of 680 patients were included of whom 340 attained ROSC. The RACA score was higher in patients with ROSC (0.62 [0.46-0.69] than in those without (0.46 [0.36-0.57]) (p<0.001). Observed against predicted ROSC indicated reasonable calibration overall (p=0.30), with better calibration in patients with a shockable initial rhythm (p=0.75) than in patients with a non-shockable rhythm (p=0.04). There was no statistical difference between observed and predicted ROSC rates in resuscitations attended by a specialist (50% vs 53%, 95% CI 45-55) or registrar (55% vs 53%, 95% CI 48-62), but rates were lower than predicted in resuscitations lead by a medical supervisor (36% vs 49%, 95% CI 25-47). Discussion: Developing a practical severity-of-illness scoring system for out-of-hospital cardiac arrest patients would allow patient heterogeneity adjustment and measurement of quality of care in analogy to commoly used severity-of-illness- scores developed for the similar purposes for the general intensive care unit population. However, transferring RACA score to another country with different population and EMS system might affect the performance and generalizability of the score. Conclusions: This study found a good overall calibration and moderate discrimination of the RACA score in a physician staffed urban EMS system which suggests external validity of the score. Calibration was suboptimal in patients with a non-shockable rhythm which may due to a local do-not-attempt-resuscitation policy. The lower than expected overall ROSC rate in resuscitations attended by medical supervisors requires further study.