Understanding neural mechanisms of strategic learning: correlates, causality, and applications

This is a systematic study on learning in the repeated game from the neuroeconomics perspective. Theoretically, learning theory has been developed to complement the traditional game theory in seeking to explain how and which equilibria might arise as a consequence of nonequilibrium dynamics among agents with bounded rationality. Empirically, learning models have been widely used to describe the evolvement of observed behavior over the course of field and laboratory experiments. While game theorists are trying to make learning theory more empirically relevant (Fudenberg and K. Levine 2009), experimentalists often found it difficult to distinguish different learning models based on behavioral choice data alone (Salmon 2001; Wilcox 2006). Here I sought to investigate learning mechanism from an alternative perspective: the neuroeconomics perspective, by combining the game theory experimental paradigm, parametric learning models, and neuroscience methods. In the first part of the thesis, I sought to identify the underlying learning rule by investigating how the brain encodes and computes learning signals used to guide behavior in a repeated normal-form game. Specifically, I combined functional neuroimaging of a multi-strategy competitive game with computational modeling of three widely used classes of learning models—reinforcement, belief-based learning, and their hybrid, experience-weighted attraction (EWA). I found evidence for distinct signals for reinforcement and belief-based learning in the brain. More importantly, I rejected the hypothesis of a hybrid EWA process at the neural level, even though it outperforms reinforcement and belief-based learning models behaviorally. Based on these findings, I hypothesized that behavioral choices are a product of a dual-system process at the brain level involving reinforcement and belief-based learning signals. Although the neural imaging method provides a new dimension of data and biologically plausible criterion for model testing, it is silent about the causal relation between brain regions and learning signals. In order to validate the neuroimaging results and establish the necessary roles of brain regions for strategic learning, I then compared ii the behavior of focal brain lesion patients to normal volunteers that are matched in terms of demographics and cognitive measures. In particular, I studied three different types of lesion patients: orbital frontal, dorsal lateral prefrontal and basal ganglia patients, which allowed me to dissociate the different roles necessarily to strategic learning. In the third part of the thesis, I applied the above findings on the neural circuitry underlying strategic learning to explore the behavioral signature of a special yet important population, the elderly individuals. In particular, I compared the behavioral results from the strategic learning under two experimental settings: playing against other intelligent players and against a computer agent; and between two populations: the healthy elderly individuals and young individuals. Our behavioral results suggest that elderly individuals adjust more slowly. Interestingly, this is not because elderly individuals are insensitive to the new experience but because their prior belief decays more slowly than young individuals. I further posited that within elderly population, their prior decays more slowly when they are playing against intelligent people than against a computer agent. This comparative study serves as a first step for developing biomarkers to quantify decision-making deficits and will shed light on the individual differences in productivity and intellectual viability often found within the elderly population