Investigating the construct validity of contemporary cognitive assessment

© 2019 Navaneetham J SudarshanWechsler Adult Intelligence Scale – Fourth Edition (WAIS-IV) is the most commonly used instrument for the assessment of cognitive abilities, and in making high-stakes decisions in Australia. The current thesis comprises four studies related to the validity of using WAIS-IV in cognitive assessment. The norms for the WAIS-IV used in Australia are based on the US standardization sample. The Woodcock Johnson III (WJ III) battery has been normed for Australian population under the age of 22. This study investigated the score equivalence of the WAIS-IV and the WJ III cognitive test battery in an Australian university student sample. One hundred and sixty-four participants, of whom 38 undertook both tests, were assigned to tests in a blocked, random order. Scores from all, and from those aged less than 22 years, were analysed. The Full Scale Intelligence Quotient of the WAIS-IV was significantly higher than the WJ III General Intellectual Ability score in every analysis. The effect sizes were medium and large in the independent and repeated measure analyses respectively. We argue for norming of the WAIS-IV, or any test used for high-stakes assessment, for Australian population to ensure fairer decisions. In the second study a five-factor model was found that fitted the WAIS-IV U.S. 15-indicator standardization samples for the nine age-groups between 16 and 69 years (N for each group = 200). A strong metric invariance for three of the five factors and a partial intercept invariance for the remaining two was established. Pairwise comparisons of adjacent age groups supported the inference that cognitive-trait group differences are manifested by group differences in the test scores. The third study sought a common model for a clinical sample of scores (N= 321) and the standardization samples. The clinical sampled differed from the standardization samples in that (a) the heterogeneous clinical sample comprised scores from patients, aged 16 to 91, with disparate neurological diagnosis in contrast to the nine age-group stratified standardization samples, (b) only the 10 core subtests were administered in the clinical sample, and (c) the clinical sample had missing data. We found (a) a difference in response strategy between the clinical and non-clinical sample, but (b) a weak factorial invariance of a five-factor measurement model across the clinical and standardization samples. We argue that the results do not disconfirm the hypothesis that the same five latent abilities found in the analysis of the 15 subtest scores in the standardization samples underlie the scores in the clinical population. In the fourth study six models from prior studies were used to generate Monte Carlo simulation data to assess sampling variation in outcomes when model misspecification is restricted to the alternate models canvassed for WAIS-IV. We found that (a) the fit of most models to samples from the different populations were good, (b) the estimated value of a parameter unrelated to the variables germane to the misspecification is true to the population value, (c) increased sample size slightly improved RMSEA, and had minimal effect on parameter estimates, (d) the average fit measures in estimations with and without warnings were not different, (e) the number of replications had minimal effect on the fit and the estimated parameter values. The results suggest that (a) post hoc modification to improve fit in a single sample is difficult to justify, (b) even in an ill-fitting model the estimates of parameters unrelated to the misspecification may be accurate, and (c) theoretical insight and heuristic reasoning aid the identification of problematic parameters