Efficiency Comparison of Unstable Transductive and Inductive Conformal Classifiers

In the conformal prediction literature, it appears axiomatic that transductive conformal classifiers possess a higher predictive efficiency than inductive conformal classifiers, however, this depends on whether or not the nonconformity function tends to overfit misclassified test examples. With the conformal prediction framework’s increasing popularity, it thus becomes necessary to clarify the settings in which this claim holds true. In this paper, the efficiency of transductive conformal classifiers based on decision tree, random forest and support vector machine classification models is compared to the efficiency of corresponding inductive conformal classifiers. The results show that the efficiency of conformal classifiers based on standard decision trees or random forests is substantially improved when used in the inductive mode, while conformal classifiers based on support vector machines are more efficient in the transductive mode. In addition, an analysis is presented that discusses the effects of calibration set size on inductive conformal classifier efficiency.Sponsorship:This work was supported by the Swedish Foundationfor Strategic Research through the project High-Performance Data Mining forDrug Effect Detection (IIS11-0053) and the Knowledge Foundation through theproject Big Data Analytics by Online Ensemble Learning (20120192).QC 20180209