Reward Function Shape.

<p>We investigate the effect of the reward function shape by changing it from a “wide” Gaussian (Panels A and B, ) to a “narrow” Gaussian (Panels C and D, ). Here we plot average error graphs as they provide a measurement that allows us to compare systems with different reward functions. To produce the average error graphs (panels A–D, left column), we have averaged over 16 independent animats performing 5 blocks of 512 trials. Every point of the graph represents the average normalised error of the system (i.e. the normalised absolute difference between the target angle and the decision angle ) which, similar to <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0018539#pone-0018539-g003" target="_blank">Figure 3</a>, is calculated over a separate block of 128 analysis trials (without updating the synaptic weights). Error bars show the standard deviation over the 16 independent animats. We observe that after 5 blocks of trials, the system corresponding to the “wide” Gaussian reward function without lateral connections shown in (A) has reached a lower final error than that of the system with lateral connections (B). When a narrow Gaussian reward is instead used, the system with lateral connections (D) recovers this difference in final error with respect to the system without lateral connections (C). As with the previous plot we show (right column panels) the gradient of the system configurations A–D by plotting the sum of the potential weight change (calculated in the same way as previously). For clarity, plots A and B are repeated from <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0018539#pone-0018539-g003" target="_blank">Figure 3</a>. We note that when a narrow Gaussian reward is used the system with lateral connections (D) learns over a very narrow band close to the target angle. In contrast the profile of the system without lateral connections (C) remains consistent with that of the wide reward, learning across a broader range of the population.