Measuring nonlinear signal combinations using EEG

This thesis has examined nonlinear signal summation using a combination of EEG and computational modelling. Nonlinearities are essential to many perceptual phenomena, but remain poorly understood beyond the earliest levels of the sensory pathways. Many nonlinear physiological phenomena, such as cross-orientation suppression (XOS), can be readily described by models of normalisation for neuronal gain control in primary visual cortex (V1). However, there are several nonlinearities that normalisation cannot fully explain. For example, super saturation – which can occur in around 17% of V1 and 25% of V2 neurons in macaque (Peirce, 2007b) – would be considered metabolically wasteful within a framework of normalisation: an over- exertion of the normalisation pool upon the excitatory response of a neuron. It seems unlikely that this non-monotonic nonlinearity does not serve a purpose. Considering this, gain control may not be the only function served by nonlinearities in the visual system (and beyond). Peirce (2007b, 2011, 2013) proposed that nonlinearities in V1 could also be used by neurons in mid-level vision to detect signal conjunctions for combinations of stimuli. This kind of signal summation would make possible neurons with more complex receptive field preferences than are commonly observed in V1. For example, neurons that are sensitive to multiple orientations and a narrow bandwidth of spatial frequencies would be useful for detecting patterns coherent plaids. However, at any one point in time, several different nonlinearities can occur in response to a stimulus. Being able to distinguish one from the other is more difficult than it might at first seem. The experiments described throughout this thesis aimed to disentangle nonlinearities, identify those that were selective for specific stimulus combinations and characterise them. In Chapter 3 we used transient electroencephalography (EEG) to measure the earliest component – C1 – of visual evoked potentials (VEPs) to brief presentations of gratings and their combinations into coherent and non-coherent plaids. By comparing the C1 response to gratings and plaids, we aimed to measure the degree of nonlinear summation taking place for coherent and non-coherent grating combinations. The outcome was inconclusive; there was limited evidence to suggest the involvement of extra nonlinearities in the processing of coherent plaids that were not involved in processing non-coherent plaids. This might be an inherent problem with the transient EEG approach. Although it produces a rich time course of data following the presentation of a stimulus, the response is the sum of many nonlinearities. To overcome this, we took an alternative approach in Chapter 4 and used the two-frequency method of steady-state EEG. This allows you to tag each of the grating components forming a plaid, as well as directly measure nonlinearities at intermodulation frequencies. We found a plaid-selective intermodulation response, which was larger for coherent plaids than it was for non-coherent plaids. In support of this representing an additional nonlinearity beyond normalisation, the degree of component suppression did not differ between coherent and non-coherent plaids for any of the grating components used. We generated a simple two-layered computational model of signal summation to try and explain the complexity of responses generated in to combinations of gratings. The model took the form of a logical AND gate, allowing it to respond selectively to conjunctions of signals. It appears that this kind of mechanism can represent well the responses we observed using EEG. It is not clear how a mechanism that makes use of saturating nonlinearities to perform selective signal summation would behave across contrast. At lower contrast levels, before many neurons reach the rising slope of their dynamic range, it might be that the mechanism fails altogether. Using a similar paradigm to Chapter 4, we measured intermodulation responses across a wide range of contrast levels in Chapter 5. We again found a selective intermodulation response that was larger for coherent plaids. However, this difference only occurred at the highest component contrast level that we used – 32%. Having found a nonlinearity in the visual system that appeared to selective for particular combinations of grating stimuli, we wanted to investigate whether similar nonlinearities are put to use in other brain regions. In Chapter 6 we generated auditory stimuli – three pure tones – that were combined to form a consonant and a dissonant chord. Substantial component suppression was observed for one of the components. However, no intermodulation responses or component-based harmonic responses were observed. Bringing these findings together, the transient approach to measuring nonlinear responses is somewhat limited, and provided only hints at what might be the presence of ‘conjunction’ detectors in mid-level vision. On the other hand, it appears that the two-frequency approach is extremely useful for measuring and disentangling multiple nonlinear responses. Here – in the visual system, at least – this was useful for distinguishing responses relating to lateral inhibition caused by the presence of multiple stimulus components from responses relating to the combination of responses relating to those stimulus components in the brain. Conjunction detectors that operate at moderate to high contrast levels appear to be present in mid-level vision. In the one auditory study that we conducted, no clear pattern of results were observed, making it difficult to assess the usefulness of the two-frequency approach in that domain