Efficient Feature Subset Selection Techniques for High Dimensional Data

ABSTRACT: A database can contain several dimensions or attributes. Many Clustering methods are designed for clustering low-dimensional data. In high dimensional space finding clusters of data objects is challenging due to the curse of dimensionality. When the dimensionality increases, data in the irrelevant dimensions may produce much noise and mask the real clusters to be discovered. To deal with these problems, an efficient feature subset selection technique for high dimensional data has been proposed. Feature subset selection reduces the data size by removing irrelevant or redundant attributes. This algorithm works in two different steps that is minimum spanning tree based clustering methods and representative feature cluster selection. The proposed Pearson correlation measure focused on minimized redundant data. As a result, only a small number of discriminative features are selected. KEYWORDS: Feature subset selection, filter method, feature clustering, graph-based clustering, correlation analysis I. INTRODUCTION In machine learning and statistics, feature selection, also known as attribute selection, is the process of selecting a subset of relevant features for use in model construction. Feature selection has been an active and field of research and development for decades in statistical pattern recognition, machine learning, data mining and statistics. It has proven in both theory and practice effective in enhancing learning efficiency, increasing predictive accuracy, and reducing complexity of learned result