Effectiveness of OER use in first-year higher education students' mathematical course performance: A case study

This chapter aims to understand the impact of Open Educational Resources (OER) on first-year mathematics students at the Instituto Profesional Providencia (IPP) in Santiago, Chile, where more than half (52%) of first-year students typically drop out of their studies. In order to address this, the institution established an innovation fund and a project to profile, assess and monitor student performance through an early warning system. IPP stakeholders envisioned that a strategy to promote OER uptake could complement these efforts. By looking at an OER intervention amongst firstyear students, this study seeks to identify ways in which OER can provide new tools, opportunities, and contexts to improve student performance and lower dropout rates by answering the following research questions: What is the effect of OER use on firstyear students’ mathematics course performance? In face-to-face instruction, what is the effect of OER use on first-year students’ class attendance? What are teachers’ and students’ perceptions of the OER adoption process? To answer questions one and two, this study used a quantitative method to estimate the effect of OER use on students’ mathematical course performance and class attendance. Five groups of first-year students were compared based on the analysis of two scenarios. In Scenario 1, a control group and two treatment groups were in a traditional face-to-face classroom setting. The control group relied on a proprietary textbook; the first treatment group was taught with the help of a Khan Academy OER collection; and the second treatment group was taught by means of a custom-designed Open Textbook. Scenario 2 compared two classes in blended-mode Algebra and Calculus courses. The control group relied on a proprietary resource, and the treatment group used a Khan Academy collection of OER in addition to the proprietary resource. In order to estimate the effectiveness of OER use on students’ mathematical performance, the impact analysis focused on three result variables: (1) students’ marks before the final exam, (2) students’ final exam marks, and (3) students’ final course marks after the exam. To answer research question three, a mixed-methods approach was applied in the form of a series of semi-structured interviews, a focus group discussion and a student survey. The students who used the Khan Academy OER collections or the Open Textbook were asked to participate in this study in order to better comprehend learners’ and teachers’ perceptions of OER. Students in Scenario 1 who used Khan Academy resources obtained statistically significantly better exam grades than those who used the proprietary resource or the Open Textbook, suggesting that not all kinds of OER have the same effect on student performance. In Scenario 2, there was no improvement in mathematical course performance amongst students using OER. In terms of student attendance, face-to-face mode students who used Khan Academy OER had significantly lower attendance levels than those who relied on the proprietary textbook, which may be due to the fact that when students have access to the infrastructure required to access OER remotely they tend to work more from home. With regard to student and teacher perceptions of the OER adoption process, the qualitative and quantitative data confirmed the assumption that OER can be relevant and useful to Chilean students. The chapter concludes with the insight that “openness” does not necessarily produce an impact in and of itself, but is instead part of a greater set of tools and practices in which many variables exert an influence. Neither the intrinsic nature of information and communication technologies nor openness are tools or instruments that can be said to result in a specific outcome. The dataset arising from this study can be accessed at: https://www.datafirst.uct.ac.za/dataportal/index.php/catalog/57