Observation Accuracy for Assessors of Work-Sample Performance: Consistency across Task and Individual-Differences Correlates

Consistency and correlates of observation accuracy were examined with videotapes of mechanics performing 2 jet-engine installation tasks. Job experts confirmed errors scripted into selected task steps. Their consensus pass/fail evaluations became target scores for evaluating observation accuracy. 79 jet-engine mechanics viewed the videotapes, made pass/fail ratings on each task step, and completed cognitive, personality, rating-style, and task-effort measures. Hit rate, false-alarm rate, and bias indexes were more consistent across the 2 tasks than in previous research on performance-evaluation accuracy. A discrimination index was less stable. The pattern of individual-differences correlates of observation accuracy was for the most part different from the pattern found in research on performance evaluation and person perception