Exploring issues in the use of grading in practice. Commissioned by NHS Education for Scotland.

Executive SummaryThis literature review focuses on exploring issues of grading in practice, including reliability, validity and the implications for mentor preparation and support. Recommendations are made to inform the National Approach Working Group in formulating guidance to Higher Education Institutions around the grading of practice.Literature reviewLiterature was collected following a systematic search and spanned across fourteen professional groups. In total a 119 articles were reviewed following application of exclusion criteria. Of these, the majority were quantitative in nature (n=66); followed by descriptive accounts or opinion based (n=28); literature reviews (n=19) and qualitative (n=6).Challenges in grading practiceGrading practice reflects the conclusion of a decision making process which indicates how well a student is progressing in respect to a standard or criteria and flags up areas where students can improve.There are a number of documented challenges when grading practice. Some of the challenges such as time available, consistency and accountability of assessors are not specific to only grading practice. Those which are specific to grading relate to validity and reliability issues of the tools used and grade inflation.Grading ToolsThe review of the literature presents a number of grading tools or systems, and we conclude that evaluation of these tools is under-developed in terms of their effectiveness, usefulness, reliability and validity. Grade inflation is a well-documented problem and there are some suggested methods (although not fully evaluated) from the literature to control this. The most promising of those appears to be the use of rubrics and focused training of assessors.RubricsCarefully constructed rubrics can ameliorate grade inflation. Rubrics are made up of three key components: clearly defined performance criteria or elements; detailed descriptions of what a performance looks like at each level (or grade) of proficiency; and a rating scale which most commonly uses a three or four points. Rubrics can be used both formatively and summatively.Page 7 of 47Training of assessors / mentorsIf nursing and midwifery were to use grading within practice, there are a number of training requirements suggested by the literature for mentors: developing an understanding and interpretation of educational terminology; how to accurately assess learning; how to assign grades; how to use the tools consistently and effectively; how to write evidence to support the graded assessment given; how to deal with borderline students; and how to deliver effective feedback. The time and methods of training and updating mentors needs to be considered.Conclusions and recommendationsFrom the literature reviewed, it is not possible to make any generalisations about grading of practice. The findings from this literature review come from a wide source of professional literature that often include limitations of being set in one geographical area, in one profession, and calculated on relatively small sample sizes. Most quantitative studies were survey design, and /or presented using descriptive statistics only. Therefore the generalisability of findings from these quantitative studies should be necessarily cautious. Only six studies demonstrated the use of a qualitative theoretical basis for the study. A number of limitations were noted and could be prone to bias. The literature review articles were mostly descriptive accounts, of for example a grading tool, or an opinion on grading, or on a developed tool. It should be borne in mind that opinion can be subject to writer bias. The level of evidence on grading of clinical practice across the literature tends to sway towards the lower end of the evidence band i.e. the usefulness, reliability, validity and effectiveness of grading of practice has still to be proven.If the grading of practice is adopted, we recommend that consideration is given to the following: The development, testing and use of rubrics; The use of rubrics for formative as well as summative assessment; The use of a multi-method approach to assessment; Comprehensive training and updating sessions for assessors; Ongoing evaluation and monitoring of the grading process used