One Tree to Explain Them All

Random forest is an often used ensemble technique,renowned for its high predictive performance. Random forestsmodels are, however, due to their sheer complexity inherentlyopaque, making human interpretation and analysis impossible.This paper presents a method of approximating the random forestwith just one decision tree. The approach uses oracle coaching,a recently suggested technique where a weaker but transparentmodel is generated using combinations of regular training dataand test data initially labeled by a strong classifier, called theoracle. In this study, the random forest plays the part of theoracle, while the transparent models are decision trees generatedby either the standard tree inducer J48, or by evolving geneticprograms. Evaluation on 30 data sets from the UCI repositoryshows that oracle coaching significantly improves both accuracyand area under ROC curve, compared to using training dataonly. As a matter of fact, resulting single tree models are asaccurate as the random forest, on the specific test instances.Most importantly, this is not achieved by inducing or evolvinghuge trees having perfect fidelity; a large majority of all treesare instead rather compact and clearly comprehensible. Theexperiments also show that the evolution outperformed J48, withregard to accuracy, but that this came at the expense of slightlylarger trees.Sponsorship:This work was supported by the INFUSIS project www.his.se/infusis at the University of Skövde, Sweden, in partnershipwith the Swedish Knowledge Foundation under grant2008/0502.