Digital approaches to researching learners' computer interactions using gazes, actions, utterances and sketches

When learners use computers, they typically look at the screen, type, use the mouse, talk, write, sketch and make gestures. This paper identifies technical, practical, ethical and methodological challenges associated with traditional methods for studying such interactions. It examines the potential of recent technologies for identifying learners’attention, recording real-time writing and sketching, and analyzing multiple data feeds in an integrated way. A study of learners’ interactions with multiple representations is used to illustrate the advantages and disadvantages of digital approaches to collecting, coordinating and analyzing observational data. The paper argues that there is a need for research into frameworks for analyzing digital data of learners’ computer interactions in systematic and principled ways