Multiple Test Batteries as Predictors for Pilot Performance: A Meta-Analytic Investigation

A Test Battery (TB) is a measurement method that is designed to assess a variety of ability constructs. The extent to which TB predicts future pilot performance has important implications for both flying organizations and applicants. The primary emphasis in the existing literature has been on scores of individual ability tests, in contrast to the scores of multiple ability tests that are typically indexed by composites derived from TBs. The selection literature lacks a focus on composite scores, and seldom links to the broad cognitive abilities that predominate TBs. The objective of this study was to investigate how the different broad ability constructs of TBs influence their predictive validities for pilot performance. Six ability groups were identified as the most common ability saturations of pilot selection TBs. On the basis of 89 studies and 118 independent samples, a series of meta-analyses were conducted to determine the criterion-related validity of the six categories of TBs for several criterions of pilot performance. The investigation revealed there was an overall small and positive relationship between TBs and flight performance. The six categories of cognitive ability TBs appeared to be valid predictors of pilot performance, and at least five of them generalize validity across studies and settings. More specifically, three sets of predictor groups were identified according to the magnitude of validity estimates. The highest validity group included Work Sample TBs (r=.34), the second highest validity group included TBs of Acquired Knowledge, General Ability, and Motor Abilities (r=.19, .18, and .17, respectively), and the lowest validity group included TBs of Perceptual Processing and Controlled Attention (r=.14 and .10, respectively). The results also indicated that there was substantial variability in the effect of cognitive abilities on flight performance, with evidence of moderators operating in most cases. Five potential moderator variables were examined that may moderate the TBs-performance relationship in flying. The analysis for the moderator variable of the number of tests in the battery (small battery/large battery), regularity of TB use in pilot selection (commonly used/uncommonly used), and criterion level of measurement (continuous/ordinal/dichotomous/ contingency table) revealed significant moderating effects on the correlations between flight performance and several types of test batteries. Other moderators related to year of publication (1987-1999/2000-2009/2010-2017) and flying organization (USAF/US Navy/Another military/Civilian) did not significantly influence the correlations between TBs and flight performance. The implications of the findings for practice are discussed, and recommendations for future research directions are provided.