Improved Apriori algorithm using power set on Hadoop

Abstract

In this era of big data, a huge volume of data is produced. Storage and analysis of such data is not possible by traditional techniques. In this paper, a good method to implement the MapReduce Apriori algorithm using vertical layout of database along with power set and concept of Set Theory of Intersection have been proposed. The vertical layout has the advantage of scanning only a limited number of records for calculating the support of an item. By use of the power set concept, we are able to generate frequent item set, in just two scans of database, reducing the complexity. The concept of set intersection is used to calculate its support. The result shows good improvement over the existing MapReduce Apriori algorithm.