Aviation Human-in-the-Loop Simulation: Best Practices for Subjective Performance Measurement

Subjective performance measurements are a useful tool for researchers and instructors to evaluate tasks that are difficult to quantify with objective data pulled from a simulator or the physiological data of pilots. Subjective performance measurements are non-intrusive measures typically conducted by human raters. Some recommendations for utilizing subjective measures include reducing the workload of the human raters, concealing the aim of the experiment from participants, utilizing multiple raters to evaluate each participant, providing raters with proper training, developing error-resistant rater forms and, including both subjective and objective measures when possible