Intelligent Typo Correction and Text Categorization Using Machine Learning and Ontology Networks

In this dissertation, we present our research work in text mining field, mainly focusing on accurately and efficiently performing the task of text preprocessing and text categorization, using machine learning techniques and ontology networks. Specifically, an innovative intelligent typo correction system, ITDC, is proposed to automatically correct misspellings in text documents using general language knowledge and domain specific knowledge extracted by machine learning algorithms. It has the capability of correcting a broad range of typos, from simple typos such as duplication, omission, transposition, substitution characters, to complex spelling errors, such as word boundary errors, unconventional use of acronyms, and multiple versions of abbreviations of the same words. It uses the generated knowledge for identifying unconventional acronyms, grouping similar words (correctly spelled and misspelled), and ranking correction candidates. An innovative text categorization model, VSM_WN_TM, is also presented. VSM_WN_TM is a special Vector Space Model (VSM) that incorporates word frequencies, ontology networks and latent semantic information. Unlike the traditional text representation using only Bag-of-words (BOW) features, it also incorporates semantic and syntactic relationship among words such as synonymy, co-occurrence and context, with the purpose of providing more inclusive and accurate text representation. The results from the performed experiments are highly encouraging. The ITDC system is evaluated through a case study that involves the automatic processing of automotive fault diagnostic text documents. The performance generated from more than 580000 automotive fault 3 diagnostic documents provided by two different automotive manufacturers show that the proposed system outperforms state-of-art spell checking systems. The proposed VSM_WN_TM model is evaluated on three publicly available datasets and one domain-specific dataset. Experiment results show that our approach significantly improves text classification by outperforming baseline approaches such as using only latent features and traditional VSM approaches.PhDInformation Systems EngineeringUniversity of Michigan-Dearbornhttp://deepblue.lib.umich.edu/bitstream/2027.42/108582/1/Yinghao Huang Final Dissertation.pdfDescription of Yinghao Huang Final Dissertation.pdf : Dissertatio