Statistical Preprocessing for Decision Tree Induction

Some apparently simple numeric data sets cause significant problems for existing decision tree induction algorithms, in that no method is able to find a small, accurate tree, even though one exists. One source of this difficulty is the goodness measures used to decide whether a particular node represents a good way to split the data. This paper points out that the commonly-used goodness measures are not equipped to take into account some patterns in numeric attribute spaces, and presents a framework for capturing some such patterns into decision tree induction. As a case study, it is demonstrated empirically that supervised clustering, when used as a preprocessing step, can improve the quality of both univariate and multivariate decision trees. 1 Introduction Decision trees have been studied widely in the statistics, pattern recognition and machine learning literature. A typical decision tree induction algorithm can be (roughly) sketched as follows. The input is a set S of n instances..