A Study of Different Pre-Processing Approaches of Text Categorization

Text Pre-processing is a process of converting raw text data in to corpus (bag of words) which is further fed into different classifiers for text categorization. This paper presents the results of an experimental study of some text pre-processing techniques used against various classification algorithms.The main intent is to understand and discover best possible pre-processing technique to procure better classifier performance. In particular, text pre-processing techniques like Document Term Matrix (DTM), Term Document matrix (TDM) and Term Frequency-Inverse Document Frequency (TF-IDF) were used against 10 different classifiers on BBC News dataset. A comparative performance analysis of classifiers is conducted using evaluation metrics like Accuracy, Precision, Recall and F-score. The results indicate TF IDF as better pre-processing method aiding better classifier performance than DTM and TDM