The Operational Effectiveness of the Behavioral Expectations Scale & the Mixed Standard Scale: A Comparative Evaluation

Performance evaluations were obtained on firefighters in a large Midwestern City. The evaluations were conducted through utilization of two different scale types (The Behavioral Expectation Scale and The Mixed Standard Scale). These evaluations were obtained in order to test the hypotheses that the MSS was psychometrically superior to the BES in the reduction of halo and leniency error and that the MSS was also the better scale type in terms of producing higher interrater reliability. Leniency error (in both the absolute and comparative sense) was examined by conducting a series of T-tests. Halo error was investigated by a comparison of the means of the dimensions from each scale. The technique used to assess interrater reliability involved estimating the reliability of the differences in the shape and level of performance profiles of firefighters. The results showed that the first hypothesis, which proposed that use of the MSS produced less halo and leniency error than did use of the BES, was supported; however, the second hypothesis, which proposed that use of the MSS produces higher levels of interrater reliability, was not supported