Assessment of Learning Through Reflecting on Doing

Over the past two decades, an emerging educational framework called ‘experiential learning’ has come to prominence in numerous fields, including medical and instructional education. Experiential learning, in contrast to rote or didactic learning, requires that the learner grapple with the various aspects of an authentic problem, engaging with the interconnected facets of the problem context, constraints, needs, and other components. This approach to learning is heavily anchored in the work of David Kolb, who posits that learning is a cyclical process in which the subject takes part in some experience, reflects on that experience, abstracts lessons identified in that reflection, and integrates the new knowledge in such a way that they may then experiment with the new framework. It is contended in this thesis that an opportunity exists to better incorporate this framework in engineering design education to enable engineering students to be more conscious, deliberative learners. In two sections of the course AME4163: Principles of Engineering Design, given in Fall 2015 and 2016 at the University of Oklahoma, the learning paradigm of “learning through reflection on doing” is embraced as the method by which students, referred to in the course as junior engineers, internalize five principles of engineering design (POED). As the students engage with the design project for the semester, they complete a series of assignments with two integrated assessment instruments, the learning statement (LS) and the material internalization inventory (MII), designed to stimulate the reflection and abstraction stages of learning identified in Kolb’s learning cycle. In this thesis, the focus is on this approach to experiential learning, in conjunction with the aforementioned assessment instruments, will enable engineering design instructors to produce graduates capable of meeting the challenges of a changing engineering landscape. Farrokh Mistree points out in his editorial for the Journal of Mechanical Design titled “Strategic Design Engineering: A Contemporary Paradigm for Engineering Design Education for the 21st Century?” that increasing complexity in engineered systems and the environments in which they are designed is prompting a need for graduates who are prepared to adapt to changing circumstances. Competence in this area requires that engineers have the ability to reflect on new information to abstract value. It is in this domain of learning that the experiential learning construct may be of most value to current and future students. Unfortunately, though experiential learning has taken root in some engineering design programs, instructors teaching such courses tend to focus on output rather than student learning. It is common practice in engineering design education for instructors of DBT courses to implement assessment strategies in which they focus on the end products of student designs: devices designed and built, programs written, results obtained, and more. In this thesis, the case is made that student learning outcomes are improved when instructors de-emphasize the importance of product “success” and instead focus on the learning acquired through the process of design itself. Furthermore, of central importance is the contention that requiring students in engineering design courses to focus on their learning process produces engineering graduates more likely to succeed in today’s engineering environment. Consequently, the following research question guiding the described research efforts in AME4163 is offered: In the context of an experiential learning engineering design, build, and test course, what are the curriculum, strategies for assessment, and tools that instructors can employ to motivate students working in team settings to learn by reflecting on doing and how can instructors characterize and assess that learning? Implicit in this question is the notion that engineering students, in practicing design, learn as a byproduct of reflection on their lived experiences. This is what is implied by the phrase “learn by reflecting on doing.” From the instructor’s need to assess this learning through reflection on doing is where this document obtains its title. Consequently, addressing this question is valuable in at least two areas: scholarly efforts to facilitate student learning in experiential settings and efforts within mechanical engineering capstone programs to produce graduates capable of meeting new industry challenges. Essential to the latter notion is the idea that graduating engineers must be capable of adapting and speculating on new competencies for themselves. It is also hoped that addressing the proposed research question and developing the necessary tools and assessment strategies will provide others with potential strategies that they might employ in similar types of design courses. Specifically, outcomes of this thesis include: * Practical alternative assessment methods used by the instructors in the Fall 2015 and 2016 iterations of the course AME4163: Principles of Engineering Design * A detailed framework for organizing and implementing novel self-assessment to enable engineering students to develop critical competencies * A comprehensive course rubric, in the form of a booklet for students, mapping the pedagogical goals to the course activities to enable connectivity It is hoped that the research outlined in this thesis will serve both engineering design educators and researchers as a roadmap for scaffolding their courses to enable students to develop the ability to adapt through learning by reflecting on doing. As this thesis contains both a framework for such courses and a thorough guide for the use of particular assessment strategies and tools, it thus may serve in a practical capacity. Further, remaining and newly-identified questions are leveraged into a proposal for a doctoral dissertation in which they will be pursued in greater detail so as to broaden the scope of this work