A Case Study Of The Impact Of A Sytematic Evaluation Process In A Graduate Medical Education Residency Program

The Accreditation Council for Graduate Medical Education has charged institutions that sponsor accredited Graduate Medical Education programs (residency and fellowship specialty programs) with overseeing implementation of mandatory annual program evaluation efforts to ensure compliance with regulatory requirements. Physicians receive scant, if any, training in program evaluation methodology. Human Performance Technology (HPT) offers models suitable for residency program evaluation as well as trained evaluators who are experts in evaluation. Leaders in the field of HPT have called for empirical studies to examine the impact of HPT models in a variety of contexts. This single case study examined the impact of using a systematic evaluation process, the Impact Evaluation Process, (Guerra-López, 2007b, 2007c; Guerra-López, 2008), as a means for annual program evaluation in an ophthalmology residency program sponsored by large healthcare institution in the Midwest. Outcome data from 2011, 2012, and 2013 (the years in which the Impact Evaluation Process, was utilized), was analyzed and compared to prior years evaluation efforts. Surveys with residency program stakeholders were conducted in 2011 and 2013. Results indicate that the number of recommendations for program improvement, types of recommendations and completed recommended actions increased in years that the systematic evaluation process was implemented. Recommendations generated using the systematic evaluation process were actionable (specific and measurable) and aligned to program deficits. Some program outcomes improved during the three years of systematic evaluation process implementation, while one performance outcome declined during this time. Stakeholder perceptions about the process indicated that anticipated and realized benefits of the process differed