DOI 10.1007/s00221-009-2060-6REVIEW Striatal action-learning based on dopamine concentration

Abstract The reinforcement learning hypothesis of dopa-mine function predicts that dopamine acts as a teaching sig-nal by governing synaptic plasticity in the striatum. Induced changes in synaptic strength enable the cortico-striatal network to learn a mapping between situations and actions that lead to a reward. A review of the relevant neu-rophysiology of dopamine function in the cortico-striatal network and the machine reinforcement learning hypothe-sis reveals an apparent mismatch with recent electrophysio-logical studies. It was found that in addition to the well-described reward-related responses, a subpopulation of dopamine neurons also exhibits phasic responses to aver-sive stimuli or to cues predicting aversive stimuli. Obvi-ously, actions that lead to aversive events should not be reinforced. However, published data suggest that the phasic responses of dopamine neurons to reward-related stimuli have a higher Wring rate and have a longer duration than phasic responses of dopamine neurons to aversion-related stimuli. We propose that based on diVerent dopamine concentrations, the target structures are able to decode reward-related dopamine from aversion-related dopamine responses. Thereby, the learning of actions in the basal-ganglia network integrates information about both costs and beneWts. This hypothesis predicts that dopamine con-centration should be a crucial parameter for plasticity rules at cortico-striatal synapses. Recent in vitro studies on cor-tico-striatal synaptic plasticity rules support a striatal action-learning scheme where during reward-related dopa-mine release dopamine-dependent forms of synaptic plas-ticity occur, while during aversion-related dopamine release the dopamine concentration only allows dopamine-independent forms of synaptic plasticity to occur