The incremental validity of a Situational Judgement Test (SJT) relative to personality and cognitive ability to predict managerial performance

Thesis (MComm (Industrial Psychology))--University of Stellenbosch, 2009.ENGLISH ABSTRACT: The last two decades have witnessed an upsurge in the research and use of psychometric tests to aid in the prediction of managerial performance. Currently the most prevailing predictor constructs of managerial performance are cognitive ability, personality, and experience. However, researchers and practitioners are still looking for ways in which to maximise the prediction of managerial performance. In recent years, Situational Judgement Tests (SJTs) have become an increasingly popular selection tool. SJTs are multidimensional psychometric instruments designed to assess an individual’s judgement concerning work-related situations. Evidence to date indicates that SJTs are valid predictors of performance, especially for managerial positions in which interpersonal interactions are important. The main objective of this study was to examine whether SJTs significantly add to the prediction of managerial performance over other measures used for managerial selection, such as measures of cognitive ability and personality. Measures of specific cognitive abilities, personality and a SJT were administered to branch managers in a South African retail bank (N = 124) to investigate the ability of the measures to predict managerial performance. Managerial performance was measured using three measures; Performance Ranking, a Behavioural Observation Scale (BOS) and an Overall Performance Rating. Hierarchical multiple regression was used to investigate the relationship between the predictor composites and the managerial performance measures. Findings reveal different prediction patterns for the three criteria. A multiple correlation coefficient of .442 (p > .05) was obtained when predicting Performance Ranking measures, .308 (p < .05) was obtained for predicting the Behavioural Observation Scale (BOS) measure, and .318 (p > .05) was obtained when predicting the Overall Performance Rating measure. Therefore, only when predicting the BOS measure, the SJT provided incremental validity over cognitive ability and personality measures. Consequently, the average of the scores of the three criterion measures, i.e., the Managerial Performance Composite, was used to evaluate the a priori hypotheses. A multiple correlation of .366 (p > .05) was obtained for predicting the Managerial Performance Composite criterion. Results therefore indicate that the SJT did not exhibit meaningful or statistically significant incremental prediction over cognitive ability and personality to predict the composite managerial performance measure.AFRIKAANSE OPSOMMING: Die laaste twee dekades het ‘n toename in die gebruik van psigometriese toetse in die voorspelling van bestuurdersprestasie waargeneem. Tans is kognitiewe vermoë, persoonlikheid en ervaring die mees algemene voorspellingskonstrukte vir bestuurdersprestasie. Navorsers en praktisyns is egter op soek na maniere om die voorspelling van bestuurdersprestasie te verbeter. ‘n Onlangse verwikkeling is dat “Situational Judgement Tests” (SJTs) toeneem in gewildheid as seleksie-metode. SJTs is multi-dimensionele psigometriese toetse wat ontwerp is om ‘n individu se oordeelsvermoë ten opsigte van werksverwante situasies te assesseer. Navorsing tot op hede wys dat SJTs geldige voorspellers van prestasie is, veral vir bestuursposisies waarin interpersoonlike interaksies belangrik is. Die hoofdoel van hierdie studie was om te ondersoek of SJTs betekenisvolle waarde toevoeg tot die voorspelling van bestuurdersprestasie bo die gebruik van ander meetinstrumente wat vir bestuurskeuring gebruik word, soos metings van kognitiewe vermoë en persoonlikheid. Vir hierdie doel, is takbestuurders in ‘n Suid Afrikaanse bank (N = 124) se kognitiewe vermoëns, persoonlikheid en situasionele beoordelingsvermoë getoets om die vermoë van die meetinstrumente om bestuurdersprestasie te voorspel, te ondersoek. Bestuurdersprestasie was deur middel van drie meetinstrumente bepaal; prestasie-rangordening (“Performance Ranking”), ‘n gedragsobservasieskaal (“Behavioural Observation Scale”) en ‘n algehele prestasiebeoordelingsmeting (“Overall Performance Rating”). Hiërargiese meervoudige regressie-ontleding was gebruik om die verhouding tussen die voorspellers en die bestuurdersprestasiemetings te ondersoek. Verskillende voorspellingspatrone is vir die drie meetinstrumente gevind. ‘n Meervoudige korrelasie koeffisiënt van .308 (p < .05) is vir die voorspelling van die BOS meting verkry, terwyl .442 (p > .05) en .308 (p < .05) onderskeidelik vir die voorspelling van die prestasie-rangordening en algehele prestasiebeoordelingsmeting verkry is. Gevolglik het slegs die BOS meting inkrementele geldigheid getoon. Die gemiddeld van hierdie drie metings se tellings is gebruik om ‘n bestuurdersprestasie-kombinasietelling “Managerial Performance Composite” te skep wat gebruik is om die finale besluit rakende die a priori hipoteses te maak. ‘n Meervoudige korrelasie van .366 (p >.05) was gevind ten einde die bestuurdersprestasie-kombinasietelling te voorspel aan die hand van die voorspellers