REVIEW Communicated by Vladimir Vapnik Regularization Theory and Neural Networks Architectures

We had previously shown that regularization principles lead to ap-proximation schemes that are equivalent to networks with one layer of hidden units, called regularization networks. In particular, standard smoothness functionals lead to a subclass of regularization networks, the well known radial basis functions approximation schemes. This paper shows that regularization networks encompass a much broader range of approximation schemes, including many of the popular gen-eral additive models and some of the neural networks. In particular, we introduce new classes of smoothness functionals that lead to differ-ent classes of basis functions. Additive splines as we11 as some tensor product splines can be obtained from appropriate classes of smooth-ness functionals. Furthermore, the same generalization that extends radial basis functions (RBF) to hyper basis functions (HBF) also leads from additive models to ridge approximation models, containing as special cases Breiman’s hinge functions, some forms of projection pur