ACh and NE: Bayes, uncertainty, attention, and learning

Uncertainty in various forms plagues our interactions with the environment. In a Bayesian statistical framework, optimal inference and learning, based on imperfect observation in changing contexts, require the representation and manipulation of different forms of uncertainty. We propose that the neuromodulatory systems such as acetylcholine (ACh) and norepinephrine (NE) play a major role in the brain's implementation of these uncertainty computations. ACh and NE have long been supposed to be critically involved in cognitive processes such as attention and learning. However, there has been little consensus on their precise computational functions. We propose that acetylcholine reports expected uncertainty; norepinephrine signals unexpected uncertainty. The interaction between these formally distinct sorts of uncertainty is suggested as playing a important role in mediating the interaction between top-down and bottom-up processing in inference and learning. The generative models we use to describe probabilistic relationships in the environment belong to the class of noisy dynamical systems related to the Hidden Markov Model (HMM). At any given time point, the animal has uncertainty about the hidden state of the world that arises from two sources: the noisy relationship between the true state and imperfect sensory observations, and the inherent non- stationarity of these hidden states. The former gives rise to expected uncertainty, the latter to unexpected uncertainty. These theoretical concepts are illustrated by applications to several attentional tasks. When ACh and NE are identified with the proposed uncertainty measures in these specific tasks, they exhibit properties that are consistent with a diverse body of pharmacological, behavioral, electrophysiological, and neurological findings. In addition, numerical and analytical analyses for these models give rise to novel experimental predictions. Preliminary data from several experimental studies engendered by this set of theoretical work will also be discussed