Nature and Validity of Complex Problem Solving

This thesis investigates the nature and validity of complex problem solving (CPS). The main focus lies on analyses of three research questions dealing with CPS’ (1) internal structure, its (2) structural stability combined with comparisons of performance differences across groups, and its (3) construct validity. In previous research, results on CPS’ (1) internal factor structure have been solely based on samples with high cognitive performance, (2) structural stability of CPS across groups has not been tested yet, and analyses of performance differences across groups have been rather scarce. Further, results on (3) construct validity dealing with the relation of CPS to other measures of cognitive performance are inconsistent. By applying a multiple task approach called MicroDYN to measure CPS, (1) internal factor structure is tested in groups of high school students, university students, and blue-collar workers, varying considerably in age and cognitive performance. Thereby, the interplay of theoretically derived CPS dimensions (use of strategies, knowledge acquisition, knowledge application) and their relation to characteristics of CPS tasks is analyzed in-depth. Further, for the first time in CPS research, (2) structural stability of CPS across groups is evaluated before performance differences are compared. Finally, (3) construct validity of CPS is investigated focusing on CPS’ incremental validity beyond reasoning in explaining school performance. In summary, the present work addresses several gaps in existing research. In chapter 1, the construct CPS and the measurement approach MicroDYN are introduced. Subsequently, previous results on internal structure, structural stability, performance differences, and construct validity are reported and followed by a brief description of the four empirical papers that are the main body of this thesis. The full papers are located in chapters 2 to 5. Whereas the first two papers are already in press after having successfully passed peer-review, the latter two papers are currently under revision. Furthermore, chapter 1 and chapter 6 refer to other papers including supplementary contributions of the author of this thesis on CPS research, which are listed as “additional papers” on page 10. The first empirical paper included in this thesis analysed internal structure and construct validity of CPS using data from a sample of university students. A 2-dimensional structure comprising the dimensions of knowledge acquisition and knowledge application fitted the data best and use of strategies as third dimension did not yield important information on CPS performance beyond knowledge acquisition. Further, CPS showed incremental validity beyond reasoning in explaining variance in school grades (cf. chapter 2). The second paper investigated structural stability and performance differences in CPS across high school students of different ages in a Hungarian sample and related performance in CPS to reasoning and parental education. Measurement invariance analyses based on structural equation models confirmed structural stability of CPS across groups and revealed that CPS performance increased with higher age. Further, CPS showed incremental validity beyond reasoning in analyses based on manifest variables. Moreover, parental education predicted performance in CPS (cf. chapter 3). The third contribution expands analyses of the second paper by comparing the structure of CPS across samples of high school students, university students, and blue-collar workers varying considerably in age and cognitive performance. As expected, CPS was measured invariant and participants in the academic track (university students, senior high school students) performed significantly better than participants in a non-academic track (blue-collar workers, junior high school students; cf. chapter 4). The fourth paper investigated structural stability across gender and nationality by comparing German and Hungarian high school student samples. CPS was measured invariant and analyses on performance differences showed that Germans outperformed Hungarians and males outperformed females. However, subsequent analyses revealed that performance differences could be partly explained by Hungarian females making less use of an efficient strategy (i.e., vary-one-thing-at-a-time) to generate knowledge. Further, influence of use of strategies as prerequisite for performance in CPS is discussed (cf. chapter 5). Chapter 6 provides a general discussion of this research. Papers consistently yielded a 2-dimensional internal structure of CPS comprising the dimensions knowledge acquisition and knowledge application. CPS was measured invariant across samples varying in age, gender, and nationality, which is a prerequisite for comparing competency across groups. Analyses of performance differences not only showed that participants with higher education performed better in CPS, but also deepened the understanding of the influence of use of strategies as prerequisite of CPS performance. Further, encouraging results on incremental validity of CPS beyond reasoning were found. After this summary of results, strengths of the papers are outlined and shortcomings combined with an outlook for future research are provided. In this respect, five issues are tackled: the operationalization of CPS, its convergent and criterion validity, relation of CPS to general mental ability, use of process data, and trainability of CPS. In summary, this thesis advances knowledge about CPS and emphasizes its usefulness as an indicator of cognitive performance in addition to traditional measures