Preliminary psychometric characteristics of the critical thinking self-assessment scale

Critical thinking skills (CTS) have been conceptualized as having six core cognitive skills as purported by the American Philosophical Association (APA) through a Delphi survey. The APA report further provided directions for teaching - learning and evaluation of these cognitive skills. This scale construction study was based on the APA critical thinking skills construct definition. Using the APA evaluation directions as a guide, this researcher developed a self assessment scale for measuring the CTS of undergraduate nursing students with the intention of assisting students in developing and improving their thinking skills. The construction of the scale was based on Cronbach’s Generalizability theory, and used Messick’s (1990) unitary concept of construct validity framework for evaluating the psychometric properties of the scale. The researcher developed 196 peer reviewed items for the Critical Thinking Self Assessment Scale (CTSAS) and the scale was subjected to experts’ ratings to establish content relevance and representativeness of the items to the construct. Seventeen experts from different disciplines reviewed the items and rated the items as 3 or 4 if the items defined the construct. Mean, Median, range and Content Validity Index (I-CVI) and Aiken’s Content Validity Coefficient (VIk) were computed to retain, modify or reject the items. The decision criteria for retaining the items included a value of VIk significant at p < 0.05, a value of I-CVI ≥ 0.75, and a range value of < 2.75 for the ‘0 to 5’ rating continuum. Statistical analysis of the item ratings resulted in reducing 196 items to 115. Following the rigorous content validation process, the 115 item CTSAS was tested through two developmental samples; one of 887 undergraduate nursing students from five Colleges of Nursing from Mahatma Gandhi University of Kerala State, India, and the second 144 undergraduate students from the College of Nursing, University of Saskatchewan, Canada. The questionnaire booklet also included an 18 item Need for Cognition Scale (NCS-SF) developed by Cacioppo and Petty (1982) for testing convergent validity of CTSAS. Exploratory data screening for the participants’ responses resulted in the deletion of four items (both the samples showed similar results in these 4 items) and 19 cases from the Indian sample, which were either missing, skewed or outliers. The remaining 111 items were analyzed for internal consistency reliability with both Indian and Canadian samples and stability reliability with the retested Indian sample (251). Exploratory Factor Analysis (EFA) using Principal Component Analysis (PCA) with varimax and oblimin rotations was run for the six core scales separately, which were classified into 16 sub scales, with the Indian sample (868). This resulted in reducing 111 items to 90 items across 14 subscales. Two of the subscales failed to emerge in EFA. The item loadings to factors demonstrated homogeneity and loaded independently with large loading weights. The items loading were mostly consistent with the pre-designated scales. The EFA retained 90 items were fixed in six path diagrams in the Analysis of Moment Structure (AMOS, added program in SPSS-PASW Statistics 18) graphics and Confirmatory Factor Analysis (CFA) was run with the 144 Canadian sample for each of the core scales to see the model fit. Three of the six core scales demonstrated acceptable goodness of fit indices and the remaining three reached almost reasonable to close fit. The Maximum Likelihood (ML) estimation-minimum discrepancy function-χ2 values were significant for all six core scales. However, the three model fit scales had a ratio of χ2 to degrees of freedom (CMIN / df) < 2 indicating good model fit. The Null hypothesis “not - close fit” (H0 = Ԑ ≥ 0.05) was rejected in favour of the research hypothesis and it may be concluded that fit of the model in the population is close (i.e., Ԑ ≤ 0.05). The fit indices for the three core scales - Interpretation, Evaluation, and Inference, strongly supported the structural fidelity of the three core scales, and it is plausible to replicate similar findings in a comparable population. The results also supported the APA critical thinking construct definition for the three cognitive skills. All the core scales revealed a reliability value ≥ 0.80 for the core scales. Some of the subscales achieved lower levels of correlation, but none were lower than 0.60. The total scale had very good internal consistency reliability; Cronbach α for the Indian sample was 0.961 and for the Canadian sample 0.975, and had high levels of communalities required for reducing the length of the scale. However, EFA and CFA gave strong results indicating further testing and analyzing the scale was necessary to refine the items. The convergent validity of the CTSAS tested with NCS-SF found significant correlations for five of the six core scales. The main limitation of the study was inadequate sample size for performing CFA. The socio-cultural influence on critical thinking was not tested. The study examined only some aspects of Messick’s unitary concept of construct validity for establishing the psychometric of the CTSAS. However, the preliminary psychometrics results of the study were very appealing and the researcher is encouraged to further examine the usability of the scale and ensuring socio-cultural relevance of the CTSAS