Mechanisms of serotonergic control in fear-related neural circuits and behaviour

Serotonin (5-HT) neurotransmission is strongly implicated in affective psychopathologies, with first-line drugs selectively targeting the 5-HT system. Further evidence for the role of 5-HT as an important regulator of emotional states comes from human gene association studies. These have identified polymorphic variants of the 5-HT transporter (5-HTT) gene with reduced transcription efficacy. Lower 5-HTT expression associates with susceptibility to affective disorders and heightened amygdala reactivity to fearful stimuli. The mechanisms by which 5-HTT gene variation leads to altered amygdala function and fear-related behaviour are unknown. The experiments presented in this thesis study these associations in tractable animal models that allow temporal- and circuit-specific control over 5-HT signalling. Specifically, a translationally relevant mouse model of 5-HTT expression variation and an optogenetic approach are used to probe how changes in 5-HT transmission shape fear-related behaviours. Using an array of behavioural tests, it was found that 5-HTT overexpressing (OE) mice exhibit a low-fear/anxiety phenotype that is already established in adolescence, indicating a possible neurodevelopmental trajectory. Acute pharmacological blockade of the 5-HTT at the time of behavioural testing did not rescue the low-fear phenotype in adolescent 5-HTT OE mice, whereas 5-HTT blockade during an early postnatal period did reverse it. These results identify a developmental origin of the fear-related phenotype associated with 5-HTT expression variation. Next, in vivo optogenetic tools were employed during a fear learning task to control neuromodulation of the basal amygdala (BA) by 5-HT neurons arising from the dorsal raphe (DRN). Photoexcitation of the DRN-BA 5-HT projection pathway enhanced fear learning and impaired fear extinction, while photoinhibition caused a fear learning deficit and facilitated fear extinction. Thus, a bidirectional functional role in fear learning for the BA-targeting 5-HT pathway was characterised. The effect of optogenetic photoexcitation of the DRN-BA 5-HT pathway on BA single unit and network activity during the fear learning task was then recorded in vivo. The study revealed that 5-HT projection activation promoted a neurophysiological state (both in terms of individual neuron activity and network oscillations) in the BA that has previously been associated with fear expression. Ex vivo electrophysiology experiments then explored the effect of optogenetic activation of the DRN-BA 5-HT pathway on the BA microcircuitry. These latter experiments identified clear evidence of 5-HT and glutamate co-transmission within the BA, which functioned in a cell type-specific and frequency-dependent manner. Finally, a combination of retrograde tracing and immunohistochemistry was applied to analyse the neurochemical identity of DRN neurons that project to the BA. The majority of these neurons were found to co-express markers for 5-HT and glutamate. To evaluate the contribution of 5-HT transmission from DRN-BA 5-HT projections to fear learning, optogenetics was combined with local infusion of 5-HT receptor antagonists. This experiment demonstrated that photoexcitation-induced enhancement of fear learning was mediated by 5-HT-dependent mechanisms. In summary, this thesis presents new evidence that DRN-BA 5-HT projections exert circuit-, temporal-, and 5-HT receptor-specific regulation of fear-related behaviour. These results highlight the importance of elucidating precise actions of 5-HT transmission in moving towards a more accurate understanding of how the 5-HT system influences emotional states.