Distributed Frequent Itemsets Mining in Heterogeneous Platforms

Abstract. Huge amounts of datasets with different sizes are naturally distributed over the network. In this paper we propose a distributed algorithm for frequent itemsets generation on heterogeneous clus-ters and grid environments. In addition to the disparity in the performance and the workload capacity in these environments, other constraints are related to the datasets distribution and their nature, and the middleware structure and overheads. The proposed approach uses a dynamic workload manage-ment through a block-based partitioning, and takes into account inherent characteristics of the Apriori algorithm related to the candidate sets generation. The proposed technique greatly enhances the per-formance and achieves high scalability compared to the existing distributed Apriori-based approaches. This approach is evaluated on large scale datasets distributed over a heterogeneous cluster