Design and evaluation of a semantic indicator for automatically supporting programming learning

How to support students in programming learning has beena great research challenge in the last years. To address thischallenge, prior works have mainly focused on proposingsolutions based on syntactic analysis to provide studentswith personalized feedback about their grammatical pro-gramming errors and misconceptions. However, syntacticanalysis falls short on informing learners how they solve theprogramming problem, even if one key learning outcome ofprogramming relates to the development of an individual'sability to solve a problem. In this article, we introduce anindicator to analyze beginners' code based on semantic prox-imity. This indicator adapts an edit distance algorithm (i.e.,the Levenshtein distance) to express the proximity of thestudents' code with the expected solution provided by theteacher, in order to express the learners' capacity to solvethe given problem. To process our indicator, we applied ma-chine learning techniques to a dataset from an introductoryprogramming course with a sample of 166 students. Thefirst results are encouraging. On the one hand, the semanticindicator can be used to automatically classify source codesas semantically correct or incorrect in 58% of the cases. Onthe other hand, the indicator is correlated with teachers'summative evaluations of students' codes. Even if furtherinvestigations must be conducted to improve the indicator'saccuracy, the results of this study make it possible to use ourapproach as the foundations for future research in semantic-based intelligent and awareness programming systems