Using Imaginary Ensembles to Select GP Classifiers

When predictive modeling requires comprehensible models, most dataminers will use specialized techniques producing rule sets or decision trees.This study, however, shows that genetically evolved decision trees may verywell outperform the more specialized techniques. The proposed approachevolves a number of decision trees and then uses one of several suggested selectionstrategies to pick one specific tree from that pool. The inherent inconsistencyof evolution makes it possible to evolve each tree using all data, and still obtainsomewhat different models. The main idea is to use these quite accurateand slightly diverse trees to form an imaginary ensemble, which is then used asa guide when selecting one specific tree. Simply put, the tree classifying thelargest number of instances identically to the ensemble is chosen. In the experimentation,using 25 UCI data sets, two selection strategies obtained significantlyhigher accuracy than the standard rule inducer J48.Sponsorship:This work was supported by the INFUSIS project (www.his.se/infusis) at the University of Skövde, Sweden, in partnership with the Swedish KnowledgeFoundation under grant 2008/0502.