The effects of a learning analytics scaffolding system in problem-based learning activities

This explanatory mixed-method study examined the effects of a learning analytics scaffolding system in supporting students and teachers in middle school science problem-based learning activities. A total of 298 middle schools 6th-grade students taught by four science teachers were grouped into three conditions: learning analytics scaffolding group (condition A), non-learning analytics static scaffolding group (condition B), and control group (condition C). This study followed an explanatory mixed-method research design, the qualitative interview data were used for interpreting and explaining the quantitative results. The statistical outcomes showed that students’ problem-solving self-efficacy in condition A is significantly higher than in the other two conditions. No main effects were found on students’ content knowledge acquisition. The student interviews revealed that the real-time support and just-in-time feedback delivered by the learning analytics scaffolding system helped them achieve higher self-efficacy in problem-solving. Moreover, students under each condition were further grouped based on gender or learning mode to explore potential differences. The statistical findings showed that male students achieved higher problem-solving self-efficacy in condition A than female students. In contrast, students in the online mode gained more content knowledge than students in the in-person mode. An important component of this study is the involvement of student-generated usage data. Both student-generated quantitative log and qualitative text data were processed and integrated with qualitative interview outcomes for understanding the quantitative survey findings. For example, survey outcomes revealed that students in the online mode in condition A achieved higher content knowledge acquisition than students in the in-person mode. The quantitative log outcomes revealed that students in the online mode in condition A accessed the scientific concepts with significantly higher frequencies and longer durations than their peers in the in-person mode, which means students in the online mode experienced a larger exposure to scientific knowledge than their in-person mode peers. Considering the findings from qualitative interviews that students in the online modes were less distracted and more likely to follow the scaffoldings, the integration of both log data and qualitative interview outcomes provided a more comprehensive picture to interpret and explain the quantitative survey results. Furthermore, all four participating teachers acknowledged the usefulness of the learning analytics scaffolding system. They indicated in the interviews that this scaffolding system enhanced students’ independence in the problem-solving process. Thus, teachers perceived larger flexibility in managing students in condition A than in the other two conditions. These outcomes indicated that the learning analytics scaffolding system supported students by providing them with more assistance and empowered teachers to facilitate problem-based learning activities in large-sized classrooms. In all, the evaluation of the learning analytics scaffolding yielded positive outcomes for both students and teachers. It supported that enhancing the problem-solving environment by embedding the learning analytics incorporated scaffolding system is a promising direction to better support students and teachers in problem-based learning activities. Last but not least, practical implications for the future implementation of learning analytics scaffolding systems were proposed.Curriculum and Instructio