Data-Mining an Online Homework System

Online homework systems are becoming increasingly popular since (when they work) they are convenient for both faculty and students. Systems that rely on mechanical grading are naturally best adapted to more mechanical types of problems, raising issues of whether an increasing reliance on such systems will privilege the assessment of procedural knowledge over the assessment of conceptual knowledge. However, online systems naturally and efficiently capture large amounts of data about student work and data-mining techniques can be applied to evaluate conceptual understanding as well as procedural understanding, even though the prompts are all procedural. In this paper, we discuss how to use detailed analysis of procedural results captured by a locally designed online homework system (tuned for the purpose of assessing conceptual understanding) to recognize conceptual growth in classes in mathematics and the likelihood of successful transfer of this understanding to later engineering classes. Patterns that demonstrate students are wrestling with new concepts and techniques for disentangling correlations in different subjects caused by successful transfer from correlations caused by general skill