Dissociation in connectionist systems

Connectionist techniques are increasingly being used to model cognitive function with a view to providing extensions, elaborations or replacements of earlier "box and arrow " models. Networks of simplified processing units loosely based on real neurons are set up with architectures based on known physiology, trained to perform appropriately simplified versions of real tasks, and iteratively refined by checking their performance against humans. Such systems can still be linked together as in the old box and arrow models, with all the old explanations of patient data carrying through, but now we can examine the details of the degradation of the various components, and removing neurons or connections constitute natural analogues of real brain damage. We can also question the validity of the old assumptions of neuropsychological inference, and explore the possibility that processing is actually more distributed and interactive than the older models implied (Bullinaria, 2002). Here I shall outline what I consider to be the essential ideas. Connectionist models learn to perform by iteratively updating their weights (e.g. by gradient descent) to minimise the output errors for appropriate training sets of input-output pairs. Adding up the network weight change contributions due to individual training patterns explains why: 1. High frequency items are learned more quickly, because the appropriate weight changes get applied more often. 2. Regular items are learned more quickly, because consistent weight changes combine while inconsistent weight changes cancel. 3. Ceiling effects arise when items are mastered. These effects are easily demonstrated in a standard feed-forward network with 10 inputs, 100 hidden units and 10 outputs. Training on two sets of 100 regular items and two sets of 10 irregular items, with one regular set and one irregular set presented 20 times more frequently than the other, results in the learning curves of Figure 1 (Bullinaria, 1999)