Using Learning Analytics to Evaluate the Effectiveness of the Flipped Classroom Approach

BACKGROUND: The Flipped Classroom seems to have been embraced globally as the latest solution to quality engineering education. Higher education institutions everywhere are extolling the benefits of this "new" innovative teaching method with many of apparently them scrambling to get on board at all costs. While the rationale for using a flipped classroom approach seems sound and reasonable, there appears to be a distinct lack of evidence showing that this new teaching and learning approach is any better than other quality teaching practices. PURPOSE: The main focus of the current study was to attempt to quantify the effects that the flipped classroom approach had on the performance of students on their course assessment tasks. DESIGN/METHOD: In this study, the flipped classroom approach was introduced for the first time to students in a second year engineering Fluid Mechanics course to try to improve student motivation and engagement, and to improve cognition and understanding of the course material. Students worked through narrated, online eLectures (recorded on Mediasite) prior to attending face-to-face class (workshop) sessions. It was hypothesised that the more time students spent working through the weekly eLecture material, the better their results would be for the course assessment items. Learning analytics was used to investigate this hypothesis. Student viewing data collected by Mediasite was analysed to determine whether there was a correlation between the total amount of time students spent on the weekly eLectures and their results for three of the summative course assessment tasks. RESULTS: The study found a poor correlation (R2=0.0498, PCC = 0.2232) between the time students spent on eLectures and the correctness of their answers to the weekly quiz questions. The correlations between eLecture time and final exam mark were equally low (R2 = 0.0688, PCC = 0.2623). CONCLUSIONS: It was proposed that the poor correlation between the time students spent on eLectures and the correctness of their answers to the weekly quiz questions may have been the result of students rushing through the eLecture material and questions “just in time” in order not to miss out on marks and to get up to speed before the workshops. However, there was no real evidence to confirm this hypothesis. It was suggested that flipped learning may only be more effective than traditional teaching practices if students work through, learn and understand the pre-lecture material properly. Student feedback on the flipped classroom method was overwhelmingly positive and clearly demonstrated that students enjoyed and embraced the new teaching and learning approach. However, this did not appear to translate into significant improvements in student cognition or deeper learning. Although the results of this initial study are generally inconclusive, and do not clearly either confirm or refute whether the flipped classroom approach was any more successful than traditional teaching approaches, the study has clearly demonstrated the intrinsic value of learning analytics as a tool to monitor student learning. While this initial study has produced some interesting and thought-provoking results, there can be many factors that influence performance and results from one student cohort to the next and these would have to be taken into account to enable more tangible conclusions