Roweis-Saul classifier for machine learning

In 2000, Saul and Roweis proposed locally linear embedding as a tool for nonlinear dimensionality reduction [1,2]. In this paper, we modify the LLE algorithm and formulate it as a classifier in a manner reminiscent of He et al [3] and name it after Roweis and Saul. Our experiments with the ORL, YALE, FERET face databases and MNIST handwritten database show that our classifier has recognition rates of 95.41%, 95.55%, 95.41% and 92.50% respectively, clearly outperforming the baseline PCA and LDA classifiers well as the recently proposed Laplacianfaces. We propose a modification to the training phase of the classifier by perturbing the within class entries of the reconstruction matrix constructed during the training phase. This perturbation leads to an increase in the success rates for some datasets. We point out the relationship between the Roweis-Saul classifier and PCA and LDA. Various hypothesis tests have been suggested for comparing classifiers [4,5]. We apply some of the hypothesis tests suggested by Dietterich and Alpaydin to compare the Roweis-Saul classifier and the Laplacianfaces and show that the Roweis-Saul classifier outperforms the Laplacianfaces for the datasets considered here