Layered Neural Nets for Pattern Recognition

Abstract-Adaptive threshold logic elements called ADALINES can be used in trainable pattern recognition systems. Adaptation by the LMS (least mean squares) algorithm is discussed. Threshold logic ele-ments only realize linearly separable functions. To implement more elaborate classification functions, multilayered ADALINE networks can be used. A pattern recognition concept involving first an “invariance net” and second a “trainable classifier ” is proposed. The invariance net can be trained or designed to produce a set of outputs that are insensitive to translation, rotation, scale change, perspective change, etc., of the retinal input pattern. The outputs of the invariance net are scrambled, however. When these outputs are fed to a trainable classifier, the final outputs are descrambled and the original patterns are reproduced in standard position, orientation, scale, etc. I t is expected that the same basic approach will he effective for speech recognition, where insensi-tivity to certain aspects of speech signals and at the same time sensitiv-ity to other aspects of speech signals will be required. The entire recognition system is a layered network of ADALINE neurons. The ability to adapt a multilayered neural net is fundamental. A new adaptation rule is proposed for layered nets which is an exten-sion of the MADALINE rule of the 1960’s. The new rule, MRII, is a useful alternative to the back-propagation algorithm