Error-Dependent Resampling for Artificial Neural Network Classifiers

this paper we investigate several ways of utilizing error-dependent resampling for artificial neural network (ANN) classifiers. The main idea of all methods is that the output error of the trained network resembles how "hard" a certain input is to learn for the network. We show how to use the network output error for speeding up leave-one-out cross-validation and training set selection for voting classifiers. The high computational costs of leave-one-out cross-validation can be drastically reduced by a simple (dirty) trick, which works surprisingly well: don't cross-validate on all patterns, but only on those with high aggregated output error. The probability that patterns with low output errors are misclassified is rather small, so we introduce almost no error (see Sections 2--5). The main objective of voting classifiers is to reduce the variance of unstable classification methods (such as trees, ANNs, . . . ). However, this can be combined with the idea of mixtures of experts, where each member concentrates on a special input region. By resampling patterns with high output error more often, we force each member of the voting committee to concentrate on patterns that were hard to learn for the earlier chosen members (see Sections 6--8). 2 Performance Predictio