Advances in Computational Mathematics 17: 331–347, 2002. 2002 Kluwer Academic Publishers. Printed in the Netherlands. Training multilayer perceptrons via minimization of sum of ridge functions

Motivated by the problem of training multilayer perceptrons in neural networks, we con-sider the problem of minimizing E(x) = ∑ni=1 fi(ξi · x), where ξi ∈ Rs, 1 i n, and each fi(ξi · x) is a ridge function. We show that when n is small the problem of minimizing E can be treated as one of minimizing univariate functions, and we use the gradient algorithms for minimizing E when n is moderately large. For large n, we present the online gradient algorithms and especially show the monotonicity and weak convergence of the algorithms