Dimensionality reduction with adaptive approximation

In this paper, we propose the use of (adaptive) nonlinear ap-proximation for dimensionality reduction. In particular, we propose a dimensionality reduction method for learning a parts based representation of signals using redundant dictionaries. A redundant dictionary is an overcomplete set of basis vec-tors that spans the signal space. The signals are jointly rep-resented in a common subspace extracted from the redundant dictionary, using greedy pursuit algorithms for simultaneous sparse approximation. The design of the dictionary is flexi-ble and enables the direct control on the shape and properties of the basis functions. Moreover, it allows to incorporate a priori and application-driven knowledge into the basis vec-tors, during the learning process. We apply our dimensional-ity reduction method to images and compare it with Principal Component Analysis (PCA) and Non-negative Matrix Factor-ization (NMF) and its variants, in the context of handwritten digit image recognition and face recognition. The experimen-tal results suggest that the proposed dimensionality reduction algorithm is competitive to PCA and NMF and that it results into meaningful features with high discriminant value. 1