Learning and Acting with Predictive Cognitive Maps

Humans and other mammals possess two remarkable abilities: the capacity to store and retrieve a seemingly boundless series of episodic memories, and the capacity to quickly make sense of and navigate their changing environments. The latter has been described as a cognitive map, and along with the capacity to store and retrieve narrative memories, has been largely localized to the medial temporal lobe. Recent theorists have suggested that these two capacities are both aspects of a single unified system of ‘experience construction.’ In such a system, complex high-dimensional sensory experiences represented in the cortex are indexed by a low-dimensional representation within the medial temporal lobe. The dynamics of this representation then allow for the generation of coherent sequences of activation which correspond to coherent narrative experiences, as well as coherent trajectories through the environment, supporting both memory and navigation. Such a theoretical perspective bears a strong resemblance to a recent class of deep neural networks called generative temporal models. In this work we explore this connection by introducing a series of increasingly complex generative temporal models, and analyzing each of their properties. We find that these models are able to learn representations which bear a strong resemblance to known representations within the medial temporal lobe, such as place and time cells. Furthermore, we demonstrate that these representations are useful for rapidly learning to perform downstream goal-directed navigation tasks using biologically plausible reinforcement learning rules. We also examine the ways in which these models can be extended to display adaptation to changes in the structure or content of the environment, a key property of the cognitive map. Finally, we compare the behavior of artificial agents utilizing these learned representations to those of humans in a complex virtual navigation task. In doing so, we find evidence that humans utilize a hybrid behavioral strategy, and that such a strategy can be modeled by artificial agents utilizing a learned place cell like representation