Toward an understanding of super-expert cognitive performance: Implications for expert systems and software engineering

The present research attempts to identify factors which account for cognitive skill acquisition at the higher end of the cognitive performance curve and understand differences between expert and super-expert skill levels on a cognitive oriented task of computer program modification. Current literature identifies automation, problem representation, learning mechanisms and cognitive abilities as factors affecting human problem solving. Super-expert programmers (rated among the top 95th percentile in their field) and expert programmers (70th to 80th percentile) performed two computer program modification tasks of enhancement of a fairly standard database program and modification of a unique and unfamiliar calendar program. A variety of performance and behavioral variables were collected from the analysis of videotapes and verbal reports on the program modification sessions. Results indicate that on the familiar task, super-experts utilized a global search strategy to obtain abstract information. Experts utilized a minimal directed search to obtain task specific information. No differences in the level of automation were found. Through factor analytic techniques, three components were identified which accounted for the solution process on the unfamiliar task. These are: Detailed Information Search, Development of an Abstract Representation and Heuristic Solution Process. Consistent with results from the familiar task, super-experts scored higher than experts on Development of an Abstract Representation for the unfamiliar task. No differences were found between the groups on the other solution components or level of task automation for the unfamiliar task. From a battery of cognitive ability tests administered to the subjects, results indicated that cognitive abilities which were significant for acquiring programming skill at the beginning levels were not related to performance at the higher programming skill levels. Implications of this difference for knowledge representation are reviewed and reappraised through implementation of elicited knowledge in an expert system shell. The results of this study have implications for: (1) training, (2) task design and (3) knowledge acquisition and knowledge representation in expert systems