Mapping the time-course and content of visual predictions with a novel object-scene associative memory paradigm

In the current thesis, we present a series of three ERP experiments investigating the time-course and nature of contextual facilitation effects in visual object processing. In all three experiments, participants studied novel object-scene pairs in a paired associate memory paradigm. At test, we presented the scene first, followed after a delay by the test object, which either matched or mismatched the scene. We manipulated two key factors. 1) In all three experiments, we manipulated the severity of contextual mismatch between the presented object and the scene, including categorical violations as well as more subtle visual distortions. In this way, we probed the level of detail at which participants were reactivating the contextually-congruent target object in response to the scene. 2) We manipulated the scene preview timing parameters both between subjects (Experiments 2.1 and 3.1) and within subjects (Experiment 3.2). Our rationale for doing this was as follows. Rather than assuming that contextual facilitation effects reflect an entirely predictive or reactive/integrative process, we tested the hypothesis that contextual facilitation was predictive in nature. If the contextual facilitation was entirely integrative (i.e., people waited until the object was presented before relating it to the scene context), we might expect that the amount of scene preview time would not modulate contextual facilitation effects. What we found instead is that allowing for additional scene preview time leads to enhanced contextual facilitation effects, suggesting that participants are using the additional time that they are observing the scene alone (beyond 200 ms, which is sufficient to extract the gist of the scene) to prepare to process the upcoming object and determine whether it matches the scene. We strengthened our findings by testing this both between subjects using only two time points, and within subjects using a parametric gradation of preview times (which also allowed us to test if our findings generalized to cases of temporal uncertainty). We also took advantage of our use of ERPs to examine dependent measures tied to specific stages of cognition. We particularly focus our analysis and discussion on contextual priming of higher-level visual features, examining how contextual congruency modulates amplitude of the N300 component under various conditions and timing constraints. We also present a set of novel visual similarity analyses relying on V1-like features, which allow us to test for context effects on visual object understanding in a component-neutral fashion. Lastly, we present analyses of context effects on other components of the waveform: the N400, as an index of semantic priming, and the LPC, as an index of response-related processing. Overall, our findings are consistent with a predictive account, in which participants use scene information to preactivate features of the upcoming object (including higher-level visual form features, as well as semantic features) in order to facilitate visual object understanding. Future work will further disentangle predictive vs. integrative processing accounts of contextual facilitation effects on visual object processing