Developing predictive models of student success in undergraduate engineering mathematics courses

Context Student success and retention are hot topics in higher education, given the strong ties to institutional reputation, funding and government policy. A recent government review into higher education found that the attrition rate for Australian universities had been relatively stagnant over the last decade, hovering at around 15% (Australian Government Department of Education and Training, 2018). This review recommended that institutions develop evidence-based risk analytics in order to enhance support strategies. Given the shifts toward online learning and increased collection of student data, there is enormous potential to harness this information for predictive models of student success. These predictive models would enable institutions to systematically identify at-risk students, understand key risk factors, and then directly target these students with tailored support interventions. Purpose This study aimed to develop models to accurately predict students at-risk of failure in two undergraduate engineering mathematics courses at the Queensland University of Technology based on data already collected within the institution relating to student background, course performance and online engagement. The key research questions were which factors are most useful for predicting student success, and how does prediction accuracy vary across a semester as additional data becomes available. Approach Data was drawn from the learning management system (LMS) and academic management system for students enrolled in the selected courses, Introductory Engineering Mathematics and Engineering Computation, in Semester 2, 2018. Binary logistic regression was used to predict students’ pass/fail outcome based on the data available at Week 0, Week 4 and Week 9 of semester. The models were evaluated using cross-validation and classification accuracy. Outcomes Progressive assessment results were found to be the strongest predictor of student success, while LMS engagement was highly significant for Engineering Computation but not Introductory Engineering Mathematics due to the weekly assessment schedule. Socio-demographic factors were not significant, but some characteristics related to students’ educational background did add value. Model performance improved over the semester, reaching 82-84% overall accuracy at Week 4 and 9. However, performance was consistently lower for the at-risk student sub-cohort. Conclusions Predictive models of student success, based on educational background, performance and engagement data can be used to identify students at-risk of course failure. Applying the models when first assessment grades become available is recommended to balance the competing requirements of prediction accuracy and early intervention. Future work should extend this study to understand how the developed models perform on subsequent cohorts