Measuring the Subjective Value of Effort in Behaviour and Midbrain Dopamine Neurons

Reward follows work, but with finite energy, all life must decide what reward is worth the work. Economic theory provides a framework of how rational decision makers optimise this choice: agents should maximise the possible utility (subjective benefit) discounted by the disutility (subjective cost) of the required effort. The dopamine reward prediction error signal is a potential neuronal substrate of this utility calculation as it is known to capture subjective value of rewards, risk and time. Other work has demonstrated pathologies of midbrain dopamine neurons and drugs that target dopaminergic signalling affect the willingness to expend effort. However, it is not clear from previous studies how effort disutility is represented in the reward prediction error signal. To resolve this question, this thesis explores the subjective value of effort through behavioural experiments in rhesus macaques and relates this to neurophysiological experiments recording from midbrain dopamine neurons in the same animals. In the first experiments, a straight-forward behavioural task is developed and validated to measure the subjective value of effort in monkeys. Random utility models are used to independently estimate the juice utility and effort disutility functions, which are then used to provide experimental evidence for the subtractive model of effort cost discounting.Extending these experiments, the second set of experiments examines whether monkeys form an internal reference point of effort expenditure and how this affects their choice. A model in which an effort reference point is learnt from previous choices is developed and used to demonstrate, that like humans, the expectation of effort has a significant effect on the subjective value of effort in a predictable manner. In a different task, effort is presented not as an absolute quantity in the options but a per-unit (or ‘wage’) quantity. By presenting options with two rewards and manipulating the wage associated with one of the rewards, this thesis demonstrates monkeys choose consistently with the principles of revealed preference theory. In recordings of midbrain dopamine neurons in the awake animal, the results of the first two sets of experiments are related to the activity of dopamine neurons during effort valuation. At the presentation of effort-predicting stimuli, a reference-dependent effort cost is encoded. Introducing risky effort cues, I elicited effort prediction errors and by comparison to reward prediction errors demonstrated a common scale of encoding between effort and reward. Taken together these findings suggest subjective effort valuation is reference-based and encoded in midbrain dopamine neurons, supporting the hypothesis that dopamine neurons encode a net utility signal.Personal funding was from the Cambridge-Rutherford Memorial International PhD Scholarship, Cambridge Trusts and Trinity College, Cambridge. This work was made possible by funding from the European Research Council and the Wellcome Trust