Phenomenological modelling of sensory integration phenomena using self-organized feature maps

The main contribution of the work presented herein is a method of transferring information from one lattice of artificial neural units to another. The essence of the present method consists of four parts: (1) the wrapping of the artificial neural lattice into a module that given an input to this lattice outputs normalized coordinates for the location of peak activity and the magnitude of the activity at that location; (2) the inter-module exchange of information being carried out by using the coordinates of the neuron that respond the strongest to the input given to the artificial neural lattice; (3) the concept of the transformation map which re-maps normalized coordinates from one module's output space on another's; and (4) fusion carried out by summation of activity fields induced by artificial neural lattices. The method, that is to been seen as an alternative to Hebbian linkage, is validated by implementation in the artificial neural network architecture used for carrying out the two cortical modelling experiments described below.It is the modelling of sensory integration phenomena that is the main theme in this thesis. The two modelling experiments used for validating the presented method examine two cases of sensory convergence; the case when the converging signals are congruent and the case when they are incongruent. Convergence of sensory signals is interesting because it is a process taking place in the brain, and because it plays an important role in perception.The sensory convergence phenomenon of main interest is that of audiovisual integration. Based on a series of reports on different aspects of letter–phoneme integration an architecture is developed, the multimodal self-organizing network (MMSON). The result is an artificial neural network whose essential building blocks are two types of modules, each employing its own learning law, none requiring manual intervention. The built MMSON architecture consists of three modules which are interconnected, and each module processes one of three different types of pre-processed stimuli: visual, auditory, and the bimodal combination of these. The architecture is generally useful; it is, for example, equally suitable for simulating the integration of different features in a single modality, and the ideas behind it are new.In the first modelling experiment the focus is on integration of congruent stimuli, leading to combination and an enhancement of perception. The main features modelled using the architecture, are the combination of unimodal signals at the bimodal level and feedback from the bimodal to the auditory level. Simulation results show that the architecture's dynamics parallel results from psychology and neuroscience: (1) the qualitative enhancement of the response to a noise-perturbed sensory signal in one modality using a congruent one from a complementary modality; and (2) the ability to use input to one modality as support for distinguishing the relevant signal from a collection of signals in another modality.The second modelling experiment considers a case when integration ``fails'' due to irreconcilable incoming signals. The main phenomena under examination is the McGurk effect. This effect occurs during the perception of audiovisual syllables when particular incongruent stimulus pairs are presented to the subject. In these particular cases the conclusion of the subject's perceptual system contains neither of the presented stimuli but instead a third stimulus is perceived; e.g. an auditory ``ba'' and a visual ``ga'' is often perceived as ``da''. We show here that this phenomenon can be successfully simulated with the MMSON architecture.Another phenomenon where integration ``fails'', which is examined in this thesis, is binocular rivalry. Although this work is not in complete resonance with with the work on audiovisual integration, the architecture used for modelling binocular rivalry is related to the MMSON architecture by sharing the same foundation; it consists of communicating lattices containing artificial neurons. The work on binocular rivalry has two foci. One is to have the architecture's dynamics to simulate key properties of binocular rivalry; the skewed unimodal distribution of dominance times, and three other well-known properties of binocular rivalry that are described by Levelt's second and fourth propositions and the ``reverse of'' Levelt's second proposition. Simulation results obtained from the model are in fair quantitative agreement with psychophysical experiments in all four aspects, using one and the same set of model parameters. The other focus is to strengthen the links between binocular rivalry and sensory integration by arguing that the former phenomenon can be seen as a variant of the latter, and by indicating how the dynamics of the two coincide.Godkänd; 2011; 20110704 (tamjan); DISPUTATION Ämnesområde: Industriell elektronik/Industrial Electronics Opponent: Professor Wlodzislaw Duch, Dept of Informatics, Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University, Torun, Poland Ordförande: Professor Lennart Gustafsson, Avd för EISLAB, Institutionen för system- och rymdteknik, Luleå tekniska universitet Tid: Måndag den 19 december 2011, kl 10.00 Plats: D770, Luleå tekniska universite