1 Multiclass Classification: Margins Revisited, Novelty Detection via Trees, and Visual Object Recognition with Sparse Features Derived from a Convolution Neural Net

This thesis comprises three nearly self contained parts. First we examine a few types of multi-class Support Vector Machine (SVM) classifiers that are typically used in applied machine learning. Unlike the original binary SVM formulation, in these classifiers the margins which are being maximized in the optimization problem do not represent distances to the decision boundaries of the final classifier. We investigate whether improvement can be obtained by employing classifiers which maximiz margins with respect to the classifier’s actual decision boundaries. Perhaps surprisingly, we will prove a theorem that negates that theory- the optimization problem solved by the unified versions (Crammer & Singer, 2001), (Weston & Watkins, 1998), obtains a solution that is identical to that of the optimization problem that maximizes margins with respect to the actual decision boundaries. In addition, we present a connection between this version and the 1-vs-1 SVM multiclass classifier. Later, we explore the use of descriptors extracted from pre-trained CNNs for image classification of new classes; in our work we addressed the sparsity o