The interaction of syntax and metaphor in gesture: A corpus-experimental approach

This dissertation is a study of metaphor in usage: metaphor in language, metaphor in gesture, and how they interact. Gesture provides a route to study both the cognition associated with language and the domain-generality of cognitive processes. While English speakers may be producing metaphoric manner verbs due to the lexicalization patterns of their language, are they necessarily thinking in terms of metaphoric manner? This is difficult to judge when looking at language alone. To answer this question, we turn to metaphoric gesture.Metaphoric gestures, in which the gesture represents the source domain of a conceptual metaphor, are well-known but under-studied (Cienki and Müller, 2008b). Iconic gestures conveying information about a motion event are known to interact with the syntactic and semantic structure of speech; speakers of languages that express manner of motion in the verb gesture differently than speakers of languages that primarily express path of motion in the verb. Metaphoric usages of motion in language – prices falling, hopes rising, time flying – also interact with the grammatical patterns of language. However, we know little about how metaphoric motion in gesture interacts with grammar. In Part One of the dissertation, I focus on metaphor in language. In Chapter 2 I propose to represent metaphors as a complex network of frames, mappings, and bindings as implemented in the MetaNet Metaphor Repository (Dodge et al., 2015). This advances the representation of conceptual metaphors to a level that interfaces more accurately with representations of frames and constructions in FrameNet (Ruppenhofer et al., 2006) and Embodied Construction Grammar (Bergen and Chang, 2005). In turn, this enables the detailed analysis of metaphors and metaphor systems, as exemplified by the Location Event Structure Metaphor (Lakoff and Johnson, 1999) case study in Chapter 3. This corpus-based study is one of the first to make use of the MetaNet method for large-scale automatic metaphor identification and annotation. This approach reveals not only how the metaphor system is evoked in language, but further illustrates the conceptual structure of the metaphor. I demonstrate that although English, as a satellite-framed language, privileges manner in its lexicalization of motion events, metaphoric English motion backgrounds manner and foregrounds path. The foregrounding of path information in linguistic realization of metaphoric motion runs counter to the privileging of manner in English lexicalization patterns. This finding lays the groundwork for the investigation of the same metaphor system in gesture.In Part Two, I focus on metaphor in co-speech gesture. I investigate metaphoric motion-evoking metaphoric gestures using two complementary approaches. Chapter 4 uses a corpus approach; I analyze a parallel corpus of video gesture data in which speakers use a motion verb either literally or metaphorically in their speech while producing a co-expressive representational gesture. To analyze the corpora, I develop a set of annotation guidelines and then demonstrate the benefits of taking an image-schematic approach to gesture analysis. I argue that the image schema is the most appropriate level of structure in analyzing the form and meaning of metaphoric gestures. Results of this image schema analysis suggest that, reflecting the English language data in Chapter 3, these metaphoric gestures emphasize path and do not represent the manner of motion.Chapter 5 is the first study to take an experimental approach to metaphoric gesture that uses non-metaphoric stimuli. Participants were given short stories about state change, such as prices decreasing or grades improving, to read and re-tell to a friend; half of the stimuli contained metaphoric language and half did not. Results from this study demonstrated the viability of this methodology in eliciting both metaphoric speech and gesture, and supported those of Chapter 4. I find that speakers are more likely produce metaphoric gestures if they are also producing metaphoric language – even if the gesture evokes a different metaphor than the speech does.I unify my analyses of metaphoric motion in speech and gesture in a multi-modal Embodied Construction Grammar analysis of both co-expressive and complementary metaphoric co-speech gestures. I represent both the meaning and form of the gesture and the meaning and form of the speech including frame structure, argument structure, and metaphoric structure. This analysis provides the first formal representation of a multi-modal utterance in a construction grammar and an innovative approach to the unification of the construction of multi-modal meaning