An Evaluation of code2vec Embeddings for Scratch

The recent trend of embedding source code for machine learning applications also enables new opportunities in learn- ing analytics in programming education, but which code embedding approach is most suitable for learning analytics remains an open question. A common approach to embedding source code lies in extracting syntactic information from a program's syntax tree and learning to merge these into continuous distributed vectors (e.g., code2vec). Code2vec has been predominantly investigated in the con- text of professional programming languages, but learning analytics are particularly important in the context of educational programming languages such as Scratch. In this paper, we therefore instantiate the popular embedding approach code2vec for Scratch programs, create three different classification tasks with corresponding datasets, and empirically evaluate code2vec on them. Our experiments demonstrate that a transfer of code2vec to the educational environment of Scratch is feasible. Our findings serve as a basis to apply code embeddings to further educational tasks such as automated detection of misconceptions of program- ming concepts in Scratch programs