Evolved Decision Trees as Conformal Predictors

In conformal prediction, predictive models outputsets of predictions with a bound on the error rate. In classification,this translates to that the probability of excluding thecorrect class is lower than a predefined significance level, in thelong run. Since the error rate is guaranteed, the most importantcriterion for conformal predictors is efficiency. Efficient conformalpredictors minimize the number of elements in the outputprediction sets, thus producing more informative predictions.This paper presents one of the first comprehensive studies whereevolutionary algorithms are used to build conformal predictors.More specifically, decision trees evolved using genetic programmingare evaluated as conformal predictors. In the experiments,the evolved trees are compared to decision trees induced usingstandard machine learning techniques on 33 publicly availablebenchmark data sets, with regard to predictive performance andefficiency. The results show that the evolved trees are generallymore accurate, and the corresponding conformal predictors moreefficient, than their induced counterparts. One important resultis that the probability estimates of decision trees when used asconformal predictors should be smoothed, here using the Laplacecorrection. Finally, using the more discriminating Brier scoreinstead of accuracy as the optimization criterion produced themost efficient conformal predictions.Sponsorship:Swedish Foundationfor Strategic Research through the project High-PerformanceData Mining for Drug Effect Detection (IIS11-0053) and theKnowledge Foundation through the project Big Data Analyticsby Online Ensemble Learning (20120192).