Flexible margin-based classification techniques

Classification is a very useful statistical tool for information extraction. Among numerous classification methods, margin-based classification techniques have attracted a lot of attention. It can be typically expressed as a general minimization problem in the form of $loss + penalty$, where the loss function controls goodness of fit of the training data and the penalty term enforces smoothness of the model. Since the loss function decides how functional margins affect the resulting margin-based classifier, one can modify the existing loss functions to obtain classifiers with desirable properties. In this research, we design several new margin-based classifiers, via modifying loss functions of two well-known classifiers, Penalized Logistic Regression (PLR) and the Support Vector Machine (SVM). In particular, we propose three new binary classification techniques, Robust Penalized Logistic Regression (RPLR), Bounded Constraint Machine (BCM), and the Balancing Support Vector Machine (BSVM). For multicategory case, we propose the multicagegory Composite Least Squares (CLS) classifier, a new multicategory classifier based on the squared loss function. We study properties of the new methods and provide efficient computational algorithms. Simulated and microarray gene expression data analysis examples are used to demonstrate competitive performance of the proposed methods