Tikhonov-based Regularization of a Global Optimum Approach of One-layer Neural Networks with Fixed Transfer Function by Convex Optimization

Abstract — Regularization is useful for extending learning models to be effective for classifications. Given the success of regularized-perceptron-based (one-layer neural network) methods, we introduced a similar kind of regularization for two global-optimum approaches recently proposed by Castillo et al, which combined the degree of freedom of using nonlinear transfer functions with the computational efficiency of solving complex problems. We focused on the two approaches that used sigmoid transfer functions. The first linear approach involved solving a set of linear equations, while the second min-max approach was reduced to a linear programming problem. We introduced regularization in such a way that the first linear approach remained linear and had a close form solution, while the second min-max approach was converted from a linear programming into a quadratic programming problem. Electroencephalography recordings were used to show how classifications could be improved. I