Empirical performance evaluation in collaborative aircraft design tasks

The overarching goal at the Integrated Design Laboratory (IDL) is to understand the mechanisms of decision making and exchanges among engineers. In this study a toolbox for the assessment of engineering performance in a realistic aircraft design task is presented. It allows for the assessment of participants in different experimental conditions. The degree of task difficulty and the amount and quality of visualization are systematically varied across conditions. Using a graphical user interface the participants' mouse trajectories can be tracked. This data together with performance evaluation of the generated aircraft design can help uncover details about the underlying decision making process. The design and the evaluation of the experimental toolbox are presented. This includes the number, specificity, and ranges of design variables that can be manipulated by a participant. The major difficulty thereby is to find a ``sweet spot'' where the task is just difficult enough, such that participants display a progress in their performance. Too easy or too difficult of a task would lead to flooring or ceiling effects, where most participants will always fail or, respectively, perform perfectly. The decisions about the aircraft design parameters are therefore based on a numerical analysis of the design space. With this analysis nonlinearities and interdependencies of design parameters are revealed. The experimental toolbox will be utilized to measure design performance of individuals and groups. The results are expected to reveal ways to support multidisciplinary collaboration