Criterion-based assessment of the safety of IMG Doctors in Australia: development of a model

Abstract Criterion –based Assessment of the Safety of IMG Doctors in Australia: Development of a Model Introduction The impact of adverse events and errors attributed to doctors within health care settings is well documented within the international patient safety literature. Adverse event related morbidity and mortality for patients admitted to hospitals in developed countries remains at 10% worldwide. Significantly, adverse event indicators linked to clinical management include infection - affecting between 40% and 70% of patients worldwide annually; clinical judgement and decision –making; and, communication and documentation errors, including medication errors. Monitoring of the competence and safety of newly employed doctors, including Standard Pathway International Medical Graduates (SP IMGs) in Australian hospitals is traditionally considered the role of their clinical Unit’s assigned supervisor in the form of ‘end of rotation’ reports. Such monitoring appears to be ad hoc, irregular, and task focussed. The thesis participant cohorts were SP IMGs who are documented to make up around 28% of the overall medical workforce and up to 70% in designated ‘areas of need’. Further studies suggest that IMGs lack familiarity with Australian disease patterns and management strategies. WHO, in seeking to provide a patient safety curriculum for medical schools, adopted the Australian National Patient Safety Education Framework (NPSEF) (ACSQHC, 2005). The work of thesis represents an application and operationalization of the NPSEF within a Workplace Based Assessment (WBA) mini-CEX framework (the Patient Safety mini-CEX or PSMC), with patient safety competence and performance assessed according to the Cambridge Model. Methods Three cohorts of IMGs (N = 107), seeking employment or recently employed in public hospitals in Victoria, participated across 476 individual or group encounters for the purpose of testing of the PSMC in OSCE, High Fidelity Simulation, and WBA settings. Experienced clinical assessors and facilitators provided interactive post encounter individual and group formative feedback. Results Analysis of the findings from the OSCE and Simulation cohorts (N = 96) suggested that between 13.4 % and 44.2 % of these cohorts were not competent across eleven of the thirteen PSMC clinical items (Competency Items or CI) common to both the OSCE and Simulation encounters. The performance of between 29.4% and 44.5% of the Simulation cohort was unsatisfactory across seven CIs within the domains of Communicating Safely, Practising Safely, and Clinical Management & Using Evidence. The WBA cohort (N = 16), whose practice was facilitated by a Medical Education Officer were competent across 12 of the 13 assessed CI. Participants actively engaged with the teaching and learning opportunities afforded by the post session interactive feedback, seen as critical to supporting participants’ future clinical practice. Reliability modelling yielded a Cronbach’s Alpha of .937 for the salient items. Conclusion and Recommendations for Further Work The PSMC appeared to be capable of facilitating assessor discrimination between varying levels of safe and unsafe clinical practice and establishing individual participant fitness or readiness to practice according to the Cambridge Model. Integration of PSMC assessment into WBA schedules for newly employed IMGs and PGY 1 & PGY 2 doctors offers supervisors and clinical departments reliable and transparent measure of the safety and competence of the junior medical workforce within the Healthcare Safety Systems Model (HSSM) proposed by the thesis. Further work to establish the reliability and utility of the PSMC using a construct aligned PSMC Patient Safety Entrustability Index (PSEI) is recommended