Performance improvement of PrePost algorithm based on Hadoop for big data

Abstract

With the blasting growth in data, uptake data mining techniques to mine association rules, and then find useful information hidden in large data has become ever more important. Several existing data mining techniques often through mining frequent itemsets draw association rules and get to relevant knowledge, but with the rapid arrival of the era of big data, traditional data mining algorithms have been impossible to meet large data's analysis needs. Lately, the PrePost algorithm has been suggested, a new algorithm for mining frequent itemsets based on the idea of N-lists. PrePost in most cases outperforms other present state-of-the-art algorithms. In mind of this, we present the HPrePostPlus algorithm. A better version of PrePost based on Hadoop, that utilization a HashMap to traverse effectively the PPC tree, and improve the process of creating the N-lists related with 1-itemsets. We combine also the characteristic of Hadoop with a view to process large data. Experience has demonstrated that HPrePostPlus algorithm is greater than the state-of-the-art methods in terms of performance and scalability.