Switching between neural modes at sequential fixations in free viewing predicts successful episodic memory

ObjectivesThe formation of episodic memories is critically determined by how we visually sample the world over time via sequences of eye movements. Nonetheless, in the neuroscience of human memory, memory encoding has almost exclusively been studied in experimental paradigms where the study material is presented in a single fixed location on the screen, and where eye movements are treated as artifacts. Thus, the neural mechanisms subserving memory construction across eye movements are virtually unknown.Research questionWhat are the neural correlates of episodic memory formation during eye movements in natural visual behavior?Materials and methodsWe developed an associative memory task in which participants encoded multiple events, each comprising distinct elements from three categories (faces, places, objects) in different locations of the screen. The spatial configuration of the element locations allowed us to separate relevant and irrelevant saccades for integrating them into a coherent event. Participants memorized the event, while their EEG and eye movements were simultaneously recorded. Episodic memory was thereafter assessed by testing retrieval of the element associations specifying each event. In the EEG analysis, we overcame the problem of overlapping EEG responses to sequential saccades in free viewing using a deconvolution approach. We segmented EEG relative to the fixation onset and examined the power of EEG signals in the theta and alpha frequency bands.ResultsHigh subsequent memory performance was predicted by theta synchronization over the centro-parietal areas during fixation intervals after saccades relevant to event integration. This may reflect the binding of elements into coherent event representation. High memory performance was also predicted by alpha desynchronization during fixations after task-irrelevant saccades. This may reflect discrimination of elements from the same category, which is necessary for successful memory encoding. Finally, high frontal theta power during fixations after scrutinizing, within-element saccades predicted high memory. This may reflect increased visual sampling of elements leading to better memory.ConclusionThus, memory formation across eye movements is characterized by three neural mechanisms that rapidly turned on and off in a saccadic sequence during event encoding. These mechanisms provide essential building blocks for the construction of episodic memory during naturalistic viewing