Geometric Interpretation of Nonlinear Approximation Capability for Feedforward Neural Networks.

http://www.springerlink.com/This paper presents a preliminary study on the nonlinear approximation capability of feedforward neural networks (FNNs) via a geometric approach. Three simplest FNNs with at most four free parameters are defined and investigated. By approximations on one-dimensional functions, we observe that the Chebyshev-polynomials, Gaussian, and sigmoidal FNNs are ranked in order of providing more varieties of nonlinearities. If neglecting the compactness feature inherited by Gaussian neural networks, we consider that the Chebyshev-polynomial-based neural networks will be the best among three types of FNNs in an efficient use of free parameters. This work is supported by Natural Science of Foundation of China (#60275025, #60121302)