Characterizing Trade-off Decisions in Student Designers

Decision-making and design in uncertain situations in which information is limited and goals are ambitious or conflicting are a part of common practices of engineers. Such abilities in engineering practices are important to develop for all students from elementary school through college and graduate school. An important aspect of decision-making in design are the trade-offs decisions designers make throughout a project. Trade-offs are a complex element of a decision, that involves weighing possible outcomes against their respective benefits and costs in areas such as aesthetics, cost, degree of safety, and various performance indicators. Making trade-off decisions is an effective design practice, and is a key dimension of successful performance. Understanding how students characterize their design tradeoffs would allow educators a better glimpse into students’ design thinking. Each chapter of this dissertation provides unique insights on students’ conceptions, performance, and explanations of trade-off decisions and the patterns of variation among these parameters. The first study investigates how students describe and prioritize design trade-off decisions through the following research questions: Do students report changes in their perceived importance of making-trade-offs after a design project, and if so, how do their conceptions change? A McNemar test was used to statistically analyze pre and post changes on a Conceptions of Design Test. I found that significantly fewer students ranked trade-offs to be a LEAST important design activity (n=746). In a thematic analysis of open-responses where students explain their ranking, I found many students did not understand either the idea or the terminology of “trade-offs”, despite the relevance of making tradeoffs to practicing designers and the use of this terminology in the Next Generation Science Standards. It is crucial for designers (including student designers) to make decisions based on the emphasis they place on particular design attributes. However, without explicit instruction on the role of trade-offs in design and how to manage trade-offs, students may struggle with linking intuitive trade-off decision making with the terminology of “balancing trade-offs” that references an explicit design strategy. The second study seeks to understand and measure how students “do” trade-offs in design. Central to this study was developing a way to depict design artifact quality that: (1) encompasses multiple complementary and competing dimension, (2) can be applied consistently and systematically, and (3) is indicative of design competency. I developed the Trade-off Value protocol which assesses design artifacts in the areas of human, technical, and economic factors. The Trade-off Value is calculated as the sum of percentile ranks of student performance relative to their class. I analyzed approximately 400 student design artifacts to test the Trade-off Value protocol and calculated Trade-off Value scores for each design to investigate patterns of variation in the quality of students’ artifacts. Results suggest the Trade-off Value Protocol is a useful tool for three reasons. First, because it is conceptually grounded in the definition of design, it provides a comprehensive way to think about the interaction of client/user priorities, design possibilities and objective measures. Second, the protocol while being systematic is also easy to use. Third, the Trade-off Value protocol represents an important feature of design competency with which beginning designers struggle. In addition, using an etic approach to thematic analysis of the student design artifacts, I identified five distinct patterns of variation in artifact quality which suggests patterns in ways students address multiple complementary and conflicting design dimensions that indicates variations in design trade-off competency. The third study provides a look at how students explain trade-offs in design and the relationship between how students “do” and explain design trade-offs. Through the context of an in-class individual design challenge, I collected data from 318 middle school students including (1) the final design products, and (2) a post-challenge design reasoning elicitation problem. I characterized patterns of student trade-offs through the Dual-Process Framework for Evaluating Student Design Trade-offs. The analysis started with the calculation of artifact quality score (called Trade-off Value) for each student and the scoring of the design reasoning elicitation problem through a content analysis of student responses. I used a cluster analysis to identify groups exhibiting distinct patterns of trade-offs as evidenced from these two scores. Results suggest that students were able to understand trade-off decisions consistent with beginning designers, and that students fit into four main patterns of their approaches in experiencing and analyzing trade-off decisions. Findings may be used by educators to understand variation in students’ trade-off profiles that might affect deeper learning. Taken together, the results from these inter-related studies with over 1,000 students highlight the variation of pre-engineering students’ design trade-offs with respect to how students value, understand the language of, enact performance of, and provide their reasoning in order to start building a theory of trade-offs in student designers. This level of understanding is critical so that teaching efforts start from an understanding of what students currently do and know in order to address opportunities to encourage important student growth in not only the direction of engineering design but more importantly as contributors in innovating through global challenges