Improving the Assessment of Student Code

Current methods for automatically grading student code have significant flaws. While methods that use test sets to determine code correctness sucessfully identify perfect or extremely flawed code, they may not effectively classify code that does not fall into one of these two categories; furthermore, they may not identify inputs that will crash improperly implemented student code. I/O based testing is also unable to identify mistakes made within a program; it can only inspect the program's outputs. Our research hopes to improve this situation by creating tools that enable instructors to more easily (and fairly) assess student code quality without costly manual review. We also hope to use the results of this research to teach students how to effectively test software. As one part of this research, we tested the symbolic execution tool KLEE on a set of student programs to determine how long it would take for KLEE to identify errors in the student code. We found that if KLEE successfully determines inputs that will crash simple student programs, it will do so in a short period of time (1 second, for the given code). We also investigated and attempted to define the characteristics of a good test set. As one simple measure of quality, we created a simple tool that determines the implications between tests in a test set. We are currently developing a first-order mutation testing tool for the C language to evaluate the quality of instructor test sets. We intend to compare test sets performance on student code to their performance on mutated code to determine whether mutation testing serves as an accurate indicator of test set performance.unpublishednot peer reviewedU of I OnlyUndergraduate senior thesis not recommended for open acces