The effect of feedback on performance in a fracture detection task (Proceedings Paper)

Four observer groups with different levels of expertise were tested to determine the effect of feedback on eye movements and accuracy whilst performing a simple radiological task. The observer groups were 8 experts, 9 year 1 radiography students, 9 year 3 radiography students, and 10 naive observers (psychology students). The task was fracture detection in the wrist. A test bank of 32 films was compiled with 14 normals, 6 grade 1 fractures (subtle appearance), 6 grade 2 fractures, and 6 grade 3 fractures (obvious appearance). Eye tracking was carried out on all observers to demonstrate differences in visual activity. Observers were asked to rate their confidence in their decision on a ten point scale. Feedback was presented to the observers in the form of circles displayed on the film where fixations had occurred, the size of which was proportional to the length of fixation. Observers were asked to repeat their decision rating. Accuracy was determined by ROC analysis and the area under the curve (AUC). In two groups, the novices and first year radiography students, the feedback resulted in no significant difference in the AUC. In the other two groups, experts (p = 0.002) and second year radiography students (p = 0.031), feedback had a negative effect on performance. The eye tracking parameters were measured for all subjects and compared. This is work in progress, but initial analysis of the data suggests that in a simple radiological task such as fracture detection, where search is very limited, feedback by encouraging observers to look harder at the image can have a negative effect on image interpretation performance, however for the novice feedback is beneficial as post feedback eye-tracking parameters measured more closely matched those of the expert