E.Y.: Manifold Learning, a Promised Land or Work in Progress

ABSTRACT In this paper, we report our experiments using a real-world image dataset to examine the effectiveness of Isomap, LLE and KPCA. The 1,897-image dataset we used consists of 14 image categories. We have used this dataset in several settings, both supervised and unsupervised, and have found it to be relatively “well behaved ” (clusters do exist in a lower-dimensional space) compared to many other real-world datasets we have used. We did not use a “harder ” database because all dimension-reduction methods would have failed miserably, and we would not be able to observe, identify, and explain the limitations of manifold learning. Tasks of image clustering and classification often deal with data of very high dimensions. To alleviate the dimensionality curse, several methods, such as Isomap, LLE and KPCA, have recently been proposed and applied to learn low-dimensional, non-linear embedded manifolds in high-dimensional spaces. Unfortunately, the scenarios in which these methods appear to be effective are very contrived. In this work, we empirically examine these methods on a realistic but not-so-difficult dataset. We discuss the promises and limitations of these dimension-reduction schemes. The rest of this paper is organized into three sections. Section 2 briefly summarizes Isomap, LLE, and KPCA. Section 3 presents the results of our empirical studies. We offer our observations and concluding remarks i