Smoothing Regularizers for Projective Basis Function Networks

l S(W;m). overlap between internal units decreases for many transfer functions (e.g. sigmoids) with increasing order of the derivative. The accuracies of the approximation thus improves with increasing input dimension, and with increasing m. This is a very nice effect, since many real problems deal with many (10 or more) input variables. 4.1 Empirical Comparisons of R(W;m) vs S(W;m) For the regularizers R(W;m) to be effective in penalizing S(W;m), an approximate monotonically-increasing relationship must hold between them. The uncorrelated internal unit assumption implies that this relationship is linear. To test for such a linear scaling, we generated a large number of randomly selected networks. For each such network, we computed the values of&lt