Uncovering Expertise-Related Differences in Troubleshooting Performance: Combining Eye Movement and Concurrent Verbal Protocol Data

This study explored the value of eye movement data for uncovering relatively small expertise-related differences in electrical circuit-troubleshooting performance, and describes that value in relation to concurrent verbal protocols. Results show that in the ‘problem orientation ’ phase, higher expertise participants spent relatively more time, had a shorter mean fixation duration, and fixated more on a major fault-related component than lower expertise participants. In the ‘problem formulation ’ part of the ‘problem formulation and action decision ’ phase, themean fixation duration of the higher expertise participants was longer. In the ‘action evaluation and next action decision ’ phase, higher expertise participants spent relatively more time than the lower expertise participants. Over the different phases, only the mean fixation duration of the higher expertise participants differed significantly. The relation between the eye movement and concurrent verbal protocol data is qualitatively described. The results are discussed in perspective of the combined value of eye tracking and concurrent reports for expertise research and instructional design. Copyright # 2005 John Wiley & Sons, Ltd. Technical troubleshooting, that is, diagnosing and repairing faults in a technical system, is considered a complex process to carry out and learn. Effective performance of a troubleshooting task requires adequate domain, system, and strategic knowledg