Does sound matter? Studies of information encoding in short-term memory of signers and speakers.

DOES SOUND MATTER? Studies of information encoding in short-term memory of signers and speakers. Summary The short-term memory (STM) span, which corresponds to the longest sequence of items correctly recalled in a specific order, represents a widely used measure of STM capacity. STM span is shorter with signs as compared to speech, a robust finding that has been documented in different languages and populations using a variety of experimental paradigms. Attempts to characterize the source of modality-specific variations in STM span have been primarily of two types. Some accounts have drawn attention to structural differences between signs and verbal stimuli, supposing that signs are more complex in their internal structure and require more processing load for simultaneous integration of different features (movement, orientation, location, handshape) and longer articulatory duration. While a few results lent support to accounts assuming structural differences, findings showing that shorter STM spans persisted even when signs were carefully matched to verbal stimuli in duration, complexity, similarity etc., weaken accounts that identify structural differences as a primary cause of the disadvantage observed with signs. A second type of accounts hinges on the hypothesis that the reduced span is an effect of modality, stemming from greater STM capacity for encoding serial information in auditory STM as compared to visuo-spatial STM. The critical role these accounts assign to the encoding of temporal order information is justified by findings showing that when the task was free recall, without a requirement of a specific order, comparable spans appeared across modalities. The findings from free recall demonstrate comparable encoding of sign and speech, and working memory tests show comparable performance across modalities. Therefore the disadvantages for signs restricted to serial order recall lend support to a hypothesis linking the reduced STM capacity for signs to sequential order information. The present study aims to contribute to the investigation of the sequential order hypothesis that associates the reduced span of sign to the limitations of visuo-spatial STM processes in encoding temporal sequences. Research has shed light on the representation of serial order used in verbal STM, revealing that positions are encoded with respect to both start and end positions. The problem addressed in the present study is the source of the span reduction for ordered recall of signs, and a first question that arises is whether the same position scheme for order encoding in STM is used for signs and speech. We can explore this by identifying the scheme used to represent the position of items in a sequence in STM for signs and compare that to previous results from STM for speech. Here, we examined the perseverations that signers of Italian Sign Language produced in an immediate serial recall tasks. Perseverations were analysed to determine whether their occurrences reflected the encoding of serial positions with respect to both edges that characterizes verbal STM. In the core experiment of the present study we presented sequences of consonants of Italian sign language (LIS) alphabet to a group of 20 deaf students of Magarotto Institute in Padua, a secondary school where the communication among students and teachers is based on LIS. Sign sequences varying in length from 4 to 7 consonants were randomly presented on a computer screen at a rate of 1 second per sign. At the end of each sequence, participants were instructed to recall the letter signs in the same order. The length of the sequences was often purposely overspan to generate errors. We analysed the perseveration error pattern using the same technique as Fischer-Baum (2010). The results showed that, compared to hearing people, deaf participants demonstrated a reduced span, however, we found evidence for the same both-edges representation of position in STM for signed stimuli, suggesting that the same scheme is used to represent position in this task as is used in the STM for speech stimuli task. A next step was repeating the same experimental paradigm only using words and not letters as stimuli. Data was collected from 20 participants. However, in that case participants tended to produce responses of the same length, that made impossible to discern between different position encoding schemes. Those data were not enough to draw any definite conclusions from. A drastic extension of the subjects pool could have solved this problem, but at the moment it was not possible. There was, however, a possible bias: since the signers are literate, can read and write and possess strong skills of lip reading, there was a possibility that some phonological information could be involved in verbal material processing. To control for that, a study has been conducted involving two groups of participants: 20 signers (Magarotto institute students) and 15 speakers (University of Padua students). Two sets of experimental words were selected: similar in sign and similar in sound. Two matching control sets were balanced to experimental sets. The task was recalling sequences of 4 words, the 4 words were selected from one of the 4 sets. The hypothesis was that if the signers use phonological information, then words similar in sound would elicit more interference and would lead to weaker recall than a matching control group. However, the results demonstrate that signers show a strong effect of interference only in case of recall of words similar in sign, and no effect of similar sound, so we can conclude that there’s no phonological information involved in signers’ processing. An extended control study was also conducted to explore the contribution of the phonological loop in positional information encoding, since many theoretical explanations of the differences between STM of signers and speakers suggest that it’s the core structure for verbal information encoding for speakers. In this study the phonological loop was artificially blocked in speakers by means of adding articulatory suppression to the task of verbal stimuli memorizing and recall. However, the results of that study show that articulatory suppression leads participants to produce a greater rate of errors, but they use the same both-edges position encoding scheme, so we can conclude that the phonological loop doesn’t play a crucial role in serial order encoding scheme construction. We may have to search further for an intermediate problem of binding the elements of the input to serial position slots, that however exist in short term memory for signs as well as for spoken material, as it has been demonstrated by our studies