Resolving dependencies during naturalistic listening

Language comprehension requires the listener to determine how non-adjacent words in an utterance relate to each other. For example, given the utterance “Mary gave the book she finished to Eleanor.”, listeners rapidly recognize that Mary finished a book and that Eleanor is receiving the book from Mary. The working memory mechanisms that process these dependencies have been associated with a number of evoked responses and oscillatory power changes in EEG [1,2,3,4]. The present study aims to test the generalizability of these prior findings in two ways: (1) we test here a broad range of dependency types, rather than assessing dependencies between noun phrases and verbs only; (2) we examine evoked responses and oscillatory power changes in an everyday setting, as our participants performed the naturalistic task of simply listening to an audiobook story. STIMULUS AND MODELING: We use an openly available dataset of N=33 EEG recordings collected while participants listen to an audiobook story [5]. The text of the story is parsed using a dependency parser [6]. With this annotation in hand, we define two sets of word-by-word metrics that dissociate two different sub-processes of working memory during dependency processing: A “storage” metric sums across the dependencies that are unresolved at each particular word—thus mimicking working memory storage demands. A “retrieval” metric sums across all dependencies that are completed at a particular word, weighted by the time passed since word encoding—thus mimicking working memory retrieval demands. We define “storage” and “retrieval” for all dependencies in the text, as well as for linguistically coherent sub-sets (i.e., filler-gap, passive, embedding, sub-categorization). EEG DATA AND STATISTICAL ANALYSIS: The raw data are divided into epochs from –0.3–1 sec around the onset of content words. Epochs and channels are visually inspected for artifacts, and ICA is applied to remove ocular signals. We analyze evoked data (0.1–40 Hz band-pass) and alpha-band (8–12 Hz) power. We then queried these data using linear regression models within subject, containing the dependency measures alongside a set of lexical and sub-lexical control variables (e.g., word frequency, sound power, word order) at each electrode and time-point. Beta coefficients from these regressions are clustered at the group level using a non-parametric permutation test. RESULTS: The evoked signal shows an early anterior negativity that varied as a function of the total retrieval demands per word. This is consistent with [7,8]. The evoked signal also shows a late anterior positivity associated with “storage” of filler-gap dependencies. In support of [4], “storage” of filler-gap dependencies also correlates with increased left anterior alpha power. These findings confirm, in the context of naturalistic story listening, three different neural indices of working memory processes that are familiar from sentence-level experimental paradigms. In doing so,they contribute evidence from an ecologically-natural domain in favor of the hypothesized link between grammatical relationships and memory processes. A supporting figure and a reference list is available at https://tinyurl.com/y2h5pckx