EXTERNAL DATA PREPROCESSING FOR EFFICIENT SORTING

Paper presents external sorting using data preprocessing. Generally, huge data of any organization possess data redundancy, noise and data inconsistency. To eliminate, Data preprocessing should be performed on raw data, then sorting technique is applied on it. Data preprocessing includes many methods such as data cleaning, data integration, data transformation and data reduction. Depending on the complexity of given data, these methods are taken and applied on raw data in order to produce quality of data. Then, external sorting is applied. The external sorting now takes the number of passes less than actual passes log B (N/M) + 1 for B – way external merge sorting, and number of Input / Outputs less than 2*N * (log B (N/M) + 1) of Input / Outputs and also involve least number of runs compared to actual basic external sorting