Convergence of Offline Gradient Method with Smoothing L1⁄2 Regularization for Two-layer of Neural Network

Copyright © 2014 ISSR Journals. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. ABSTRACT: In this paper, we study the convergence of offline gradient method with smoothing ⁄ regularization penalty for training multi-output feed forward neural networks. The monotonicity of the error function and weight boundedness for the offline gradient with smoothing ⁄ regularization. the usual ⁄ regularization term involves absolute value and is not differentiable at the origin. The key point of this paper is modify the usual ⁄ regularization term by smoothing it at the origin are presented, the convergence results are proved, which will be very meaningful for theoretical research or applications on multi – output neural networks