Towards simple, easy-to-understand, yet accurate classifiers

Abstract. For better interpretability of class structure in data we want to use Support Vector Machines (SVM) for exploratory data analysis. This is easier to do when data is linearly separable. However, when data is not linearly separable, the results of SVM classifiers with non-linear kernels are more difficult to understand, partly due to the mapping to a higher dimensional space. In this paper, we design a method for weight-ing linear support vector machine classifiers or random hyperplanes, to obtain a classifier whose accuracy is comparable to the accuracy of a non-linear support vector machine classifier, and whose results can be readily visualized. We conduct a simulation study to examine how our weighted linear classifiers behave in the presence of known structure, compared to support vector machines with non-linear kernels. We describe the results of our simulation study on 2-class non-linearly separable data, where the data sets are generated by varying the shape of the clusters, and vary-ing the number of variables. The results show that the weighted linear classifiers might perform well compared to the non-linear support vector machine classifiers, and they are more readily interpretable than the non-linear classifiers. The normals to the separating hyperplanes are viewed using rotations or tours of the data.