Watch and Learn?! Five studies into the use and effectiveness of captioned video for L2 listening comprehension and vocabulary acquisition.

This thesis, which is organized around five empirical research studies, aims at investigating how video can be enriched in order to support second language (L2) learners’ listening comprehension and the cognitive processes that are involved in L2 vocabulary acquisition. In this dissertation, we focus on the use and effectiveness of different types of L2 subtitles or captioning (i.e. both the on-screen text and the video are presented in the L2) for vocabulary acquisition and listening comprehension.The introductory chapter, which presents the theoretical grounds, the research objectives, and the main methodological choices, is the first of seven chapters included in this thesis. The theoretical framework consists of two parts. In the first part, we describe the aspects involved in the comprehension of audio-visual input. The second part of the theoretical section describes the aspects involved in vocabulary acquisition and the cognitive processes needed for vocabulary acquisition to take place. We also focus on the incidental/intentional learning distinction, which is methodological in nature (Hulstijn, 2001, 2003) and refers to telling or not telling students that vocabulary tests will follow the video. This is also referred to as test announcement.In chapter 2, we present the first study, i.e. a meta-analytic review focusing on the effectiveness of captions with respect to the two linguistic benefits studied in the present dissertation: vocabulary learning and listening comprehension. Rather than presenting a comprehensive narrative summary of previous research, we argue that the accumulation of research on captioned video necessitates a thorough synthesis that aims at investigating the overall effectiveness of captioning for content comprehension and vocabulary learning. The main advantage of a meta-analysis is that it enables us to answer research questions that evolve around the overall effectiveness of a treatment or the potential moderating effect of study variables (e.g. type of test used and participants’ proficiency level) on the treatment’s effectiveness. The results of the meta-analyses show overall large effect sizes demonstrating the overall superiority of full captioning for enhancing comprehension and vocabulary learning gains. The moderator analyses reveal that type of listening test moderated the effect sizes. This initial study also enables us to make a number of informed research decisions and indicates that research on captioning should add new elements, at the level of captioning visualisations and at the level of comprehension and vocabulary testing.The main objective of this thesis is to study how captioning can be presented in order to support comprehension processes and stimulate vocabulary acquisition. In four consecutive experimental studies, we test the effect of four types of captioning on listening comprehension and vocabulary acquisition: (1) keyword captioning (only the words that are essential to the meaning are presented in the captioning line); (2) full captioning and highlighted keywords; (3) glossed keyword captioning (students can access the meaning of the keywords while watching the videos). We compare these new types of captioning to (4) full captioning and (5) a control group without captioning. We also investigate the effect of test announcement on vocabulary learning and the use of captioning.This thesis aims at answering four main research questions: (a) What is the effectiveness of 1) type of captioning for listening comprehension and vocabulary acquisition and 2) test announcement for vocabulary acquisition; (b) What is the effect of type of captioning and test announcement on learners’ use of captions?; (c) What is the relation between the use of captioning and the effectiveness for vocabulary acquisition?; (d) How do L2 learners perceive the usefulness of captioning? In order to answer these research questions, we conducted four experimental studies. The results of these studies are presented in chapters 3 to 6.Chapter 3 presents an exploratory study on the effectiveness of captioning types (1), (2), (4), and (5) for listening comprehension and vocabulary acquisition (N = 133). The results of this study reveal that students with captioning score significantly better than students with video only on subsequent vocabulary tests. However, the captions do not substantially contribute to students’ making form-meaning connections. No differences are found between the groups on the comprehension tests.Chapter 4 builds on the results of chapter 3 but exclusively focuses on listening comprehension (N = 226). This study compares scores of students with subtitles (1) or (4) to a control group (5) on a comprehension test consisting of global and detailed questions. Students with full captioning (4) score significantly higher on the global comprehension test than students in groups (1) and (5). This study also investigates how students perceive the usefulness of captioning types (1) and (4). Overall, students are positive about full captions but they are negative about the value of keyword captions for enhancing video comprehension.Chapter 5 (N = 51) exclusively focuses on vocabulary and studies whether students pay attention to unknown words in captioning types (1) and (4). This study also investigates whether test announcement enhances students’ word retention and their use of captions. This study uses eye-tracking technology in order to gauge caption reading and quantify the time spent on unknown words. Results indicate that students pay attention to unknown words and that keyword captions as well as test announcement may be conducive to longer fixations on the unknown words. Students also indicate that keyword captions are useful for vocabulary learning from video.Chapter 6 builds on chapters 3 and 5 and investigates the effectiveness of types (1), (4), and (5) for vocabulary acquisition (N = 227). Glossed keyword captions (3) are added to the design of this study in order to verify whether access to meaning stimulates the acquisition of form-meaning connections. This study also investigates the effect of test announcement and examines whether test announcement impacts learners’ use of the glossed captions. Results indicate that the students with glossed captions make substantially more form-meaning connections than the other groups. The relationship between the use of glossed captions for a given word and the learning of that word is directly positive.The conclusion, presented in chapter 7, discusses a series of methodological choices and offers a comparison of the results across the studies. We also discuss the main pedagogical implications and provide a series of future research perspectives.status: publishe