Physical computing in primary and secondary education : techniques and tools for teaching and assessment

Computer science is becoming an increasingly important part of primary and secondary education. This change is backed by a societal shift towards a digital economy. The European Commis- sion wants 100% of key public services to be online and 90% of small- and medium-sized businesses to reach at least a basic level of digital intensity. It is clear that the integration of computer science in education is important in order to reach these goals. Nevertheless, this is not a trivial task. There are many unknowns about how to do this effectively, moreover, these unknowns are distributed across multiple areas of policy and practice. In this dissertation, we identify some of these unknowns and formulate some possible solutions. We mainly focus on supporting teach- ing practice but also formulate insights which can be helpful for making policy decisions. Since the field of computing education is very broad, we focus on the integration of programming in a physical computing context for learners ages 10 to 14. To conduct research in this field, we first created a visual programming environment for physical systems. This environment is based on existing literature and is used as a tool throughout this work. It allows learners to build and program robots both in the real world and in simulation. This strong link between the physical and virtual word allows the environment to be used in many different learning contexts, allowing us to conduct a variety of in classroom experiments. Using the environment we created, we have shown that not all ways of integrating programming into a physical computing class are equally effective. Specifically, our results show that having learners create a program from scratch is preferable to having them find errors in existing programs. Our metrics indicate that finding errors in an existing program has more risk of causing ex- traneous cognitive load in learners. Cognitive load theory states that the human mind has a limited working memory. In order for effective learning to take place, the capacity of the working memory should not be exceeded. Even though fixing a program only required learners to change one element in the code, the act of understanding what the code does, applying error detection strategies, and deducing what they can do to fix the issue, causes more cognitive load than constructing a fully functional program from scratch. As a result, learners in the fix group get stuck more often, lose motivation more quickly, tend to use trial-and-error solution strategies more often, and score lower on programming tests. Since effective teaching not only depends on the teaching method but also on the teacher, we have developed methods for support- ing teachers in their efforts to integrate programming into their teaching. To identify the most pressing needs teachers have, we set up a qualitative experiment during one of the professional de- velopment programs we created. By observing and interviewing teachers, we identified two main challenges teachers encounter. 1) The ability and willingness to create, reuse and share edu- cational materials. 2) How to assess the coding ability of their learners especially in an open learning context. We aimed to address these issues by designing and developing different tools and techniques for supporting teachers. Our ef- forts include the design and implementation of a learning object repository, a tool for sharing learning content on the web, as well as a framework for the assessment of code. The learning object repository defines a unified way of creating and sharing educational content. It solves the issues of granularity, heterogene- ity, reuse, personalization, and sharing. We argue that having a uniform system for defining learning content is important to facilitate personalized learning. The assessment framework we developed can be used by teach- ers to get insight into the aspects of code that can be assessed, giving them a grip on how to give quality feedback. It focusses on feedback of visual programs to learners ages 12 to 14. The teachers for these age groups need the most support since they usually have to teach multiple subjects and are less of an expert in each of the topics they teach. To develop this framework, we identified the gap between the existing literature about the assessment of code and how teachers perform the assessment in practice. Consequently, its strength is the link between theory and practice. The framework consists of eight main categories: 1) functionality 2) readability 3) programming concepts 4) algo- rithmic thinking 5) testing and debugging 6) mathematical con- cepts 7) creativity 8) personal aspects. Each of these categories is then split up into concepts relevant in the context of visual programming. Comparing this framework to what teachers as- sess in practice has shown that teachers are mostly unaware of these categories. This often results in incomplete, inaccurate, or invalid feedback. Additionally, the range of concepts being considered by teachers seems to grow with teaching experience. Nevertheless, even more experienced teachers are often not fully aware of which exact concepts they are assessing. Finally, we describe a method to assist teachers with their assess- ment of code by visualizing the solution paths of learners. This requires techniques to extract educationally relevant informa- tion from code. Consequently, we developed different machine learning based techniques for extracting didactical information from code and for visualizing solution paths in our programming environment. Our system extracts structural and functional in- formation from programs and embeds it into a vector space. The structural information is extracted using a Gaussian vertex edge histogram kernel. This kernel defines the structural similarity between two abstract syntax trees which can be used to cluster similar programs together. To extract functional information, we executed the programs in a simulated environment and captured the internal program state at each time step. By transforming these state variations in time into both frequency and amplitude components, we are able to create a vector representing the func- tion of a program. These vectors can be used as input for several machine learning algorithms. This allows us to cluster similar programs together and create visualizations based on the similarity of solution paths. This proof of concept application lays the groundwork for automated feedback systems