A multi-institutional, multinational study of programming concepts using card sort data

This paper presents a case study of the use of a repeated single-criterion card sort with an unusually large, diverse participant group. The study, whose goal was to elicit novice programmers' knowledge of programming concepts, involved over 20 researchers from four continents and 276 participants drawn from 20 different institutions. In this paper we present the design of the study and the unexpected result that there were few discernible systematic differences in the population. The study was one of the activities of the National Science Foundation funded Bootstrapping Research in Computer Science Education project (2003)