Alopex: A stochastic parallel optimization algorithm

A research project is described in which theoretical investigations and applications research on stochastic optimization methods based on the Alopex algorithm were carried out. The Alopex procedure is shown to be universal and efficient means of determining the conditions for maxima or minima of functions of many parameters. Alopex uses cross correlations between change in the cost function and the changes in the parameter values to determine the parameter updates at each iteration. All parameter values are updated simultaneously. This makes it ideal for implementation on parallel computer architectures. The process is stochastic, making use of noise to provide escape from secondary extrema. Various forms of the Alopex algorithm are described in detail, along with its relationship to other stochastic optimization processes. Efficacy of the method has been demonstrated by use of computer simulations for a large variety of practical problems, some involving highly non-linear functions and up to a thousand parameters. Among the problems we have studied successfully in extensive computer runs, are: curve fitting and polynomial expansions, pattern recognition, traveling salesman problem (TSP), many particle systems (crystal growth). When applied to the TSP, this method proceeds from interior of the hypercube to the vertex representing the final solution. A model of visual perception is proposed in which sensory information received at a given level in the visual pathway, is modified, by the Alopex process, to maximize responses of central pattern analyzers. The system exhibits characteristics of feature enhancement and suppression, pattern completion, and removal of ambiguities. Analyzers are connected by an association matrix which defines how the appearance of one pattern affects the sensitivities of other pattern analyzers. In this way dynamics is introduced in this model of perception. Alopex is a neuromorphic process based on our present understanding of the biological nervous system, and is readily implemented by neuronal circuitry. It has greatest potential in areas such as pattern recognition where many hypotheses are pursued in parallel, high computation rates are required, and the current best systems are far from equaling human performance