Picturing the physics behind equations and graphs: a grounded cognition based model for multimedia learning and its application in physics education

This thesis tries to answer a fundamental question in physics education: How does the design of instructional representations affect the process of constructing physics knowledge? This question is important for the creation of instructional materials of any form, ranging from printed textbooks to blackboard writings in the classroom. It is especially critical for the creation of computerized multimedia lectures, as the visualization power of the computer opens up almost limitless possibilities to represent physics concepts in novel ways. To answer this question, I bring together knowledge from three different areas: physics education research (PER), multimedia learning (MML) theory, and most importantly, the perceptual symbols system (PSS) framework of grounded cognition. I argue that neither the existing PER theories nor the existing MML models are able to provide a satisfactory answer to this question alone. The reason of which, I believe, is that these theories are based on an amodal symbol view of cognition. The PSS framework, however, “grounds” human cognition in “modal symbols”: neural activation of sensory/motor modals of the brain. By adopting this framework, I have constructed a new cognitive model for physics learning from multimedia representations that has much greater predictive power compared to the existing models, especially with respect to the effectiveness of visual representations. This new model predicts that the perceptual features of instructional representations (graphs, equations and text), can have a significant impact on students’ learning outcome. If correctly designed, perceptual features can greatly improve the effectiveness of instructional materials. We examined the major predictions of the model in two clinical experiments. The results of experiment 1 shows that perceptually enhanced design based on the new model has a positive impact on students’ conceptual understanding, as well as on their ability to transfer the knowledge learned to a different context. The results of experiment 2 suggest that perceptually enhanced design may also improve knowledge activation and facilitate the creation of multi-step solutions. However, several other factors not included in this model may also have a significant impact on the learning outcomes. None of the existing models of MML are able to account for these results. In the last chapter, we discuss several factors of the learning process that are not covered in the current model, and point out several possible directions for future improvements