Interacting bodies: Multimodal resources for the organisation of social interaction

This talk deals with “multimodality” as comprising a rich array of verbal and embodied resources that are situatedly mobilized by the participants for the organization of meaningful and publicly recognizable actions within social interaction. Focusing on face-to-face co-present interactions, documented by video recordings of naturally occurring activities in their ordinary settings, the basic starting point of the talk is the observation that participants do not only speak together, but also gesticulate and move their bodies in meaningful and coordinated ways. Gesture studies have shown that gesture in conversation originate by the same process that produces words (Kendon, 1980; McNeill, 1985). Made predominantly by speakers, but strongly oriented to their partners (Schegloff, 1984), gesture are finely synchronized with the emerging structure of talk (Kendon, 2004) and finely coordinated with the conduct of others (Goodwin, 1981). If gesture, gaze, and facial expressions have been studied within a rich interdisciplinary tradition, contemporary studies in social interaction have shown the relevance of other embodied resources (see Streeck, Goodwin, LeBaron, 2011), such as body postures (Schegloff, 1998), embodied manipulations of objects (Goodwin, 2007), bodily arrangements within space (Mondada, 2009) and body movements (Haddington, Mondada, Nevile, in press). The talk develops this enlarged vision of multimodality, and focuses on how a range of resources (gesture, gaze, body posture, body movements, walking, and embodied manipulations of artefacts) are mobilized in a concerted way within the finely tuned and methodic coordination of action between the participants. More particularly, the talk focuses on the temporal organization of these resources – their synchronicity, simultaneity, mutual adjustments, and sequentiality – showing how they are collectively mobilized, building complex emergent multimodal Gestalts. The talk also focuses on the situated way in which these multimodal resources are selected and adjusted to the specificities of the ecology of action. In this sense, the talk discusses the very notion of ‘resource’ for the organization of social interaction and the way in which multimodal resources are made locally relevant and are intersubjectively oriented to by the participants building in real time the accountability of their actions