A Multimodal Layer Perspective

Semiotic technologies manifest semiotic resources through differently configured interfaces where the “media” and the user interchangeably transform what is perceived (Ravelli & van Leeuwen, 2018; Vigild Poulsen et al., 2018). The semiotic resources are necessary learning resources that have recently become upscaled in significance as different technologies are frequently used for meaning-making purposes. Although many agents are rather confident in using various technologies, the shift in semiotic resources poses several challenges for meaning-making practices (PanMeMic, 2020). Mainly, interpretation efforts demand a recognition of the semiotic shifts of differently configured interfaces as well as how the resources are reshaped from their meaning-potentials and affordances in cognitive processing and newly prompted into the social space through the actors’ meaning-making (Kress, 2010). This creates complexity and multiplicity that variously shapes the prerequisites for meaning-making and constitutes the semiotic activity system. The presentation will illustrate how the semiotic shifts can be identified by tracing semiotic resources, with a focus on sign-systems within the multimodal layer framework (ML) (Schnaider et al., 2020). The MLs define sign-systems as the connector between the multimodal nature of composite interfaces and the multimodal character of meaning-making that shifts through technological activation and cognitive processes of actions and sign-making. The five MLs - technologies, technologies’ functional properties and semiotic properties, modes of representation, and activities – have been used in educational settings as a tool for analysis but apply to any environment to understand how sign-systems transfer across human and technological processes.lic