Stimulus-reward learning and generalization in structured environments

It has been suggested that the brain organizes knowledge about the relationships between positions in space and non-spatial regularities in a cognitive map. Such a representation of events and knowledge may facilitate goal-directed behavior by enabling the generalization of information across related states, but the neural and computational mechanisms underlying such map-based generalization are not known. Here, we combine a virtual reality task with computational modeling and functional magnetic resonance imaging (fMRI) to investigate how humans generalize across related states to infer reward values that were never directly experienced. In this task, spatial relationships between stimuli predict reward relationships in a subsequent choice task. We find that participants not only update the stimulus-reward associations they experience directly, but they also use their knowledge about the relationships between stimuli to predict values of stimuli which were not directly sampled. This behavior can be captured by a generalizing Gaussian process model which operates over a cognitive map emerging from individual exploration behavior rather than a cognitive map reflecting true Euclidean distances. Using fMRI adaptation, we further demonstrate that an experience-based, but not a Euclidean cognitive map, is represented in the hippocampal-entorhinal system. Together, this demonstrates that relational knowledge organized inhippocampal maps can be used to extrapolate across related states and thereby facilitate novel inference