Grading student work: using think aloud to investigate the assessment practices of university lecturers

Marking is important. The grades we give students and the decisions we make about whether they pass or fail coursework and exams are at the heart of our academic standards. "Markers are the gatekeepers for university quality" (Smith & Coombe, 2006:45). Assessment in higher education involves highly decentralised and subject-specific decision-making given credence in the UK context by processes of quality assurance ranging from national agencies and review systems to local teaching team practices in marking and moderation. This quality assurance is currently firmly located within a paradigm of accountability, explicit learning outcomes, constructive alignment (Biggs & Tang, 2007), transparency and criteria-based marking (Quality Assurance Authority, 2006). Whilst much of the assessment system is placed in the public domain, subject to scrutiny and debate, there remains considerable mystery surrounding the marking activity by lecturers that underpins the whole apparatus (Yorke, Bridges & Woolf, 2000). In the assessment of open-ended coursework assignments in higher education Sadler (2009a) identifies anomalies in the apparently systematic appraisal process when using explicit criteria and argues that holistic appraisals using expert judgement offer considerable potential and deserve more rigorous empirical study. This pilot study uses think aloud protocols, asking assessors to verbalise their thinking, to investigate the marking practices of university lecturers as they grade and provide feedback on written coursework assignments. It asks the question, how do lecturers make judgements about student work and what is the role of assessment criteria within that process? The use of think aloud on „live‟ marking of student work presents some ethical risks but is intended to shine some light into the mystery of university marking. This paper presents the emerging findings of the pilot project and begins to evaluate the think aloud method as an approach to investigating the marking practice of university lecturers