Understanding Pilots’ Explanations of Automation Surprises

Automation surprise may result from inadequate or mistaken “mental models” of the automation (Sarter and Woods, 1995). To study pilots’ mental models of their automation, 202 airline pilots were asked to explain five different events involving unexpected behaviors of aircraft automation. Pilots’ abilities to correctly explain the behavior of the automation differed systematically across the scenarios. The number of complete and correct responses varied from 19% to 86%, depending on the scenario. As the complexity of the automation increased, understanding decreased. Performance on the scenarios was not related to flight experience, automation experience, or source of automation training. But pilots’ conceptions of automation were related to performance on the scenarios. Implications for training are discussed