Stochastic Aware Random Forests — A Variation Less Impacted by Randomness

The impact of random choices is important to many en-semble classifiers algorithms, and the Random Forests is particularly sensible to pseudo-random number gen-eration decisions. This paper proposes an extension to the classical Random Forests method that aims to re-duce its sensibility to randomness. The benefits brought by such extension are illustrated by a large number of experiments over 32 different public data sets. The effectiveness of ensemble classifiers for classifica-tion tasks in the machine learning area is a known fact. Classical methods as Bagging (Breiman 1996) and Ran-dom Forests (Breiman 2001) are widely spread in both re-searchers and practitioners communities. However, all en-semble classifiers rely on pseudo-random choices to gener-ate randomly altered versions of the original data sets (Ban-field et al. 2007), and those choices may have a non-negligible impact on accuracy, as was demonstrated for Bag-ging and Boosting (Fernandes, Lopes, and Ruiz 2010). The data mining research area is rich in propositions to enhance well establish methods by means of efficient al-gorithms to implement classical methods, or by proposing extensions to the methods themselves. Examples of effi-cient algorithm efforts are the works of Kulkarni (2012), as well as all the implementations within the WEKA software tool (Hall et al. 2009). Different approaches are related to extend existing meth-ods by empirical changes. Guo and Fan (2011) present an extension to ensemble classifiers methods that is based on selection of classifiers according to Ensemble Pruning via Based-Classifier Replacement (EPR). Their technique aims to reduce ensemble size (less classifiers) while increase ac-curacy. DeBarr and Wechsler (2012) work is an applied ef-fort that uses some popular ensemble classifiers to spam de-tection, but it also proposes an extension, called Random Boost, that adds to the traditional Random Forests some Logit Boost techniques (e.g., weighting on classifiers). ∗Corresponding Author. The order of authors is merely alpha-betical. Paulo Fernandes is partially funded by CNPq-Brazil (P