Developing the accredited postgraduate assessment program for Fellowship of the Australian College of Rural and Remote Medicine

Introduction: Accreditation of the Australian College of Rural and Remote Medicine (ACRRM) as a standards and training provider, by the Australian Medical Council (AMC) in 2007, is the first time in the world that a peak professional organisation for rural and remote medical education has been formally recognised. As a consequence, the Australian Government provided rural and remote medicine with formal recognition under Medicare as a generalist discipline. This accreditation was based on the ability of ACRRM to meet the AMC's guidelines for its training and assessment program. Methods: The methodology was a six-step process that included: developing an assessment blueprint and a classification scheme; identifying an assessment model; choosing innovative summative and formative assessment methods that met the needs of rural and remote located medical practitioner candidates; 21 rural doctors and academics developing the assessment items as part of a week-long writing workshop; investigating the feasibility of purchasing assessment items; and 48 rural candidates piloting three of the assessment items to ensure they would meet the guidelines for national accreditation. Results: The project resulted in an innovative formative and summative assessment program that occurs throughout 4 years of vocational training, using innovative, reliable, valid and acceptable methods with educational impact. The piloting process occurred for 3 of the 6 assessment tools. Structured Assessment Using Multiple Patient Scenarios (StAMPS) is a new assessment method developed as part of this project. The StAMPS pilot found that it was reliable, with a generalisability coefficient of >0.76 and was a valid, acceptable and feasible assessment tool with desired educational impact. The multiple choice question (MCQ) examination pilot found that the applied clinical nature of the questions and their wide range of scenarios proved a very acceptable examination to the profession. The web based in-training assessment examination pilot revealed that it would serve well as a formative process until ACRRM can further develop their MCQ database. Conclusions: The ACRRM assessment program breaks new ground for assessing rural and remote doctors in Australia, and provides new evidence regarding how a comprehensive and contemporary assessment system can work within a postgraduate medical setting