Training Layered Perceptrons Using Low Accuracy Computation

A fundamental limitation of the use of error back propagation (BP) in the training of a layered feed forward neural network is the high degree of required computational accuracy. For each iteration, the weights typically change at low significant digits. Thus, layered perceptrons cannot be trained with error back propagation using low accuracy analog circuitry. Since ana-log implementations of layered perceptrons perform quite fast in comparison with their digital counterparts, this is indeed unfortunate. The training of the layered perceptron, however, is simply a minimization search in weight space to which any one of a number of search algorithms can be applied. Certain other search approaches do not require the computational precision needed by error back propagation. In this paper, we demonstrate the random search approach to training layered perceptrons can be performed using low accuracy computational precision and, therefore, can be implemented using analog computational accuracy. In spite of numerical stability, random search techniques suffer from ever increasing search time as dimensionality grows. In response, we introduce a modified random search technique (Improved Bidirectional Random Optimization- IBRO) to improve the search accuracy per iteration. Consequently our scheme will reduce overall search iterations dramatically. We compare the performance of IBRO with that of the bidirectional random optimization (BRO) method through simulations.