Data Partitioning Approach To Frequent Pattern Mining

Data mining uses algorithms to extract knowledge from a large and complex data set. Due to the large volume of data, efficiency has become an important issue in data analysis. Two basic aspects of computing including algorithmic (time and space complexity) and performance (effectiveness and efficiency) will be considered for new data mining methods. During the last decade, many techniques have been developed to deal with data mining problems. However, because the computational resources are limited, most of the state-of-the-art techniques face the difficulties and drawbacks when mining with very large and complex input data sets. In our research, we focus our considerations on a data partitioning approach to tackle the scalable challenges of frequent pattern mining. The basic idea of the data partitioning approach is quite simple. The data set is initially partitioned into fragments and the whole process is divided into two main phases. At the first phase, the data mining methods are applied to process individual fragments (in isolation from the other data fragments) to get a local result. At the second phase, a combinated step generates the global result. The main technical challenge is to achieve the same computational results (as would be executed on the entire data set) but to gain some efficiency features from the adopted data partitioning approach. Hence, the main research question is how to partition the data set so that data partitioning will be beneficial. Firstly, we present our research of a data partitioning approach to frequent itemset mining, one of the most commonly performed data mining tasks. In particular, we will consider frequent itemset mining and sequential pattern mining where substantial effectiveness benefits have been gained. We will also discuss some further potential efficiency improvements that can be gained by our pre-processing data techniques, such as incremental clustering algorithm. The next step is to extend our research into the formulation of research problems, including more advanced optimisation of the currently successfully applied procedures and deployment of the data partitioning concepts into the sequential pattern mining. It seems very natural to keep data in some well-defined sectors that should assist in the identification of efficient maintenance methods. Because a database is often dynamic in reality, maintaining sequential patterns (over a significantly long period) becomes essential for sequential pattern mining. We study incremental sequential pattern mining. We propose a new improvement to overcome the drawbacks of current incremental mining techniques. In the final part of our work, we extend our scope of research to structure mining such as tree mining and graph mining, particularly presenting the data partitioning approach to graph mining. We also discuss the completeness and correctness of the other well-known approaches to graph mining