Lexical and lip-reading information as sources of phonemic boundary recalibration

Listeners can flexibly adjust boundaries between phonemes when exposed to biased information. Ambiguous sounds are particularly susceptible to being interpreted as certain phonemes depending on the surrounding context, so that if they are embedded into words, the sound can be perceived as the phoneme that would naturally occur in the word. Similarly, ambiguous sounds presented simultaneously with videos of a speaker’s lip movements can also affect the listener’s perception, where the ambiguous sound can be interpreted as the phoneme corresponding with the lip movements of the speaker. These two forms of phonetic boundary recalibration have been demonstrated to be utilized by listeners to adapt in contexts where speech is unclear, due to noise or exposure to a new accent. The current study was designed to directly compare phonemic recalibration effects based on lexical and lip-reading exposures. A specific goal was to investigate how easily listeners are able to follow alternating lexical and lip-reading exposures, in order to determine the most optimal way in which listeners can switch between the two. In the experiment, participants (N=28)were exposed to blocked presentations of words or videos embedded with an individually determined, ambiguous token halfway in between /oop/ or /oot/. In lexical blocks, the stimuli consisted of audio recordings of Dutch words that ended in either /oop/ or /oot/, with the naturally occurring ending replaced with the ambiguous token. In lip-reading exposure blocks, the stimuli were made up of video recordings of the same native Dutch speaker pronouncing pseudo-words that visually appeared to end in /oop/ or /oot/, but the audio of the ending was also replaced with the same ambiguous token. Two types of presentations were administered to two groups of 14, with one version switching the modality of exposure after every block, and the other every four blocks. Following each exposure block, a 6 item post-test was presented, where participants heard the ambiguous token and its two neighbors from a 10-step continuum in isolation, each presented twice, and were asked to report if each sound resembled /oop/ or /oot/. Results from a mixed-factor ANOVA determined that subjects could flexibly adjust phoneme boundaries, as there was a main effect of the phoneme being biased, such that there was a greater proportion of /oot/ responses (pooled across all post-test items) following /oot/ bias blocks than following /oop/ bias blocks, F(1,28) = 15.828, p<0.01. There was also a main effect of exposure type, comparing lexical and lip-reading exposures, F(1,28) = 4.405, p<0.05 which indicated that recalibration strength was stronger following lip-reading exposure than lexical exposure. Additionally, a significant interaction between exposure type and phoneme bias was revealed, F(1,28) = 6.475, p