AN EYE TRACKING REPLICATION STUDY OF A RANDOMIZED CONTROLLED TRIAL ON THE EFFECTS OF EMBEDDED COMPUTER LANGUAGE SWITCHING

The use of multiple programming languages (polyglot programming) during software development is common practice in modern software development. However, not much is known about how the use of these different languages affects developer productivity. The study presented in this thesis replicates a randomized controlled trial that investigates the use of multiple languages in the context of database programming tasks. Participants in our study were given coding tasks written in Java and one of three SQL-like embedded languages: plain SQL in strings, Java methods only, a hybrid embedded language that was more similar to Java. In addition to recording the online questionnaire responses and the participants\u27 solutions to the tasks, the participants\u27 eye movements were also recorded using an eye tracker. Eye tracking as a method for software development studies has grown in recent years and allows for finer-grain information about how developers complete programming tasks. Eye tracking data was collected from 31 participants (from both academia and industry) for each of the six programming tasks they completed. Unlike the original study, we were unable to find a significant effect on productivity due to the language used or whether they were a native English speaker. However, we did find the same effect of participant experience on programming productivity which indicates that more experienced programmers are able to complete polyglot programming tasks in a more efficient manner. We also found that all participants looked at the sample code the same percentage of the time for a given task regardless of their experience or language variant they were given. The top level navigation behavior also remained largely unchanged across experience or language variants. We found that professionals performed more transitions in the code between the Java code and method parameters than their novice counterparts. Overall, we found that the level of polyglot programming did not have as significant of an effect as the task itself. The high-level strategy that participants employed appeared similar regardless of language variant they were given. Adviser: Bonita Shari