Feedback, agency and analytics in virtual learning environments – creating a case for change

Assessment and feedback are two important components of learning and teaching and areas of concern for the UK HE sector, which is increasingly driven by external factors including the NSS and the Teaching Excellence Framework. HE institutions (HEIs) have been particularly concerned with the efficiency, accessibility and transparency of the assessment process. Online assessment is seen as one solution to these challenges. HEIs are adopting technologies for online submission, marking and feedback in aiming to reduce administrative costs and make this process time effective and transparent (Bausili, 2018). It is therefore unsurprising that institutions are supporting the standardisation of online assessment - with policies featuring anonymous marking, moderation, online submissions, grading time and the use of rubrics (Newland et al., 2013). However, this standardisation has been suggested to negatively impact students’ perception of feedback. Winstone et al. (2017) identified barriers that inhibit the use of online feedback by students, ranging from difficulties with decoding terminologies, to their unwillingness to expend effort on reading feedback. She argues for less standardisation and more student agency in feedback to mitigate these barriers and allow more dialog and ownership. Zimbardi et al. (2017) found that when online tools are designed with the purpose of creating agency in the feedback, students do engage with it and perform better. However, one can argue that online tools are designed to support standardisation and not to enhance dialog and agency. This research examines these issues through involving users to better establish their requirements. The project comprises of a review of the literature and current technical provision of assessment and feedback in VLEs; and participatory design workshops. The workshops focus on collecting data from two HEIs in the UK using ‘Sandpits’ with students and lecturers. These look to clarify the role of VLEs in assessment and feedback, through understanding students’ perceptions of feedback and how they are being addressed and understanding teachers’ perceptions of the constraints they face. We are exploring what is available, looking to improve interface designs and features, and present these to VLE product designers. The ALT-C audience is one sensitive to the issues raised in using technologies to increase the efficiency, accessibility and transparency of assessment and may offer an additional perspective. In this workshop participants will experience a lightning review of the research to date and asked to participate in ‘sandpits’ where they will be able to critique and redesign two scenarios (‘Agency in feedback’ and ‘Analytics and transparency in feedback’) aligned to the theme 1 ‘students’ data and learning analytics’. ‘Sandpits’ is the method being used to collect data. A ‘Sandpit’ is a type of creative design-thinking focus group where participants are stimulated by a narrative of a scenario around the use of a product, object or artefact and are encouraged to critique, discuss and re-design it (Frohlich, Lim and Ahmed, 2014; Casanova and Mitchell, 2017) In closing we will reflect on participants’ redesigns and how these complement current work and add perspective to recommendations that will be made to VLE providers