Cross Validation and MLP Architecture Selection

The pe l fomnce of cross validation (CV) based MLP architecture selection is examined using 14 real world problem domains. When testing many different network architectures the results show that CV is only slightly more likey than random to select the optimal network architecture, and that the strategy of using the simplest available network architecture pe~orms better than CV in this case. Experimental evidence suggests several reasons for the poor pelformance of CV. In addition, three general strategies which lead to significant increase in the performance of CV are proposed While this paper focuses on using CV to select the optimal MLP architecture, the strategies are also applicable when CV is used to select between several difJerent learning models, whether the models are neural networks, decision trees, or other types of learning algorithms. When using these strategies the average generalization performance of the network architecture which CV selects is significantly better than the pelfonnance of several other well known machine learning algorithms on the data sets tested 1