Reflective Writing: Quantitative Assessment and Identification of Linguistic Features

Reflective writing is used in higher education to encourage students to reflect on their learning or practice; in this context, reflective writing is frequently assessed. However, not enough is known about the criteria that educators use when assessing reflection. Many studies have developed rubrics for reflective writing assessment but these instruments are based on broad qualitative criteria and make no attempt to quantify depth of reflection. In addition to this, little is known about the linguistic structure of reflective writing; although some recent studies have examined its features, these characteristics have not been correlated to levels of reflective depth. Also of concern are the ways in which technology can impact the reflective writing and assessment process. Although many studies tout the benefits of using reflective blogging in higher education there is a dearth of empirical data that compares the effectiveness of blogging with more traditional ‘offline’ journals. A Delphi study was conducted with a group of international Reflective Practice experts to determine the criteria they use to assess reflective writing. The first round of the study identified 12 indicators of reflection; in the second round experts ranked these indicators according to depth of reflection. The reflection indicators form the basis of an instrument that can be used to assign a quantitative score to a piece of reflective writing based on the depth of reflection it contains. This instrument was used to perform a content analysis on the academic reflective writing contained in 27 reflective blogs and journals. The linguistic resources used in students’ writing were also assessed. A strong correlation was found between the overall reflective score and the total number of linguistic resources used, showing that reflective writing tends to be linguistically richer. In addition to this, relationships were seen between specific features of reflective writing and levels of reflection. A reflective scoresheet was developed that maps reflective writing assessment criteria to expected linguistic resources. This new understanding of the structure of reflective writing also has implications for the fields of automated writing evaluation and intelligent tutoring. The study also compared the writing in blogs and journals and found no significant difference in the linguistic resources they utilised, showing that the type of language used is the same, regardless of medium. A correlation was seen between the provision of feedback and improvement in reflective writing over time. A model of the reflective writing and assessment process that describes how blogs can be used to support the provision of regular formative feedback was presented