Plot of state-action prediction error, simulated LC spiking and behaviour during 100 trials of the go/no-go task, (for agents with a fixed value of model decay parameter <i>α</i> not linked to state-action prediction error).

Each point within the task is assumed to last 1s and is associated with a single state-action prediction error. In (a) the raw prediction error is extracted for t = 2, when the animal receives a cue (this is the error between t = 1 and t = 2) and t = 3 when the animal receives feedback on its response to the cue (the error between t = 2 and t = 3). Because the prediction error explicitly evaluates differences between update cycles, there is no error available for the first time point. Each trial has therefore been collapsed to two time points, each lasting 1 second. In (a) the occurrence of the ‘go’ cue causes strong peaks in prediction error. This is converted into a simulated LC firing rate in (b). To visualise LC firing, a firing probability p is calculated for each second using the state-action prediction error (SAPE) as the input into a logistic function, so that where k = 8, and m is as above. Each second was then further split into 0.1s bins, during which the unit generated a single spike with probability p. This gives a physiologically reasonable [1,22,23] maximum firing rate of 10hz if p = 1. This is converted into a simulated LC firing rate in (b), showing phasic LC activation when the ‘go’ cue is heard. Plot (c) is a graphical representation of behaviour during the task at times t = 2 and t = 3 for each trial, in which the position of the coloured block describes the agent’s location and the colour shows the agent’s observation after moving.