Finding Mult ivariat e S lits in Decision Trees Using Function Optimization

We present a new method for top-down induction of decision trees (TDIDT) with multivariate binary splits at the nodes. The primary contribution of this work is a new splitting criterion called soft entropy, which is continuous and differentiable with respect to the pa-rameters of the splitting function. Using simple gradi-ent descent to find multivariate splits and a novel prun-ing technique, our TDIDT-SEH (Soft Entropy Hyper-planes) algorithm is able to learn very small trees with better accuracy than competing learning algorithms on most datasets examined. The process of finding a splitting function at a node of a decision tree is a search problem, and we choose to view it as unconstrained parametric function op-timization over the space of hyperplane weight vec