A Taxonomy and Survey of Data Partitioning Algorithms for Big Data Distributed Systems

Abstract

Data partitioning is a backbone of distributed systems that boost the performance of big data applications, especially in distributed systems. In past years, many data partitioning algorithms have been developed which had improved the big data management and its processing for the real-time applications of the big data stores. Furthermore, the feature of “elasticity” to the data partitioning has removed the need for human interaction while handling the big data applications on the distributed system during the high workloads and skews. In this survey, a taxonomy is proposed that characterizes and classifies various types of data partitioning algorithms, which will help to identify the current limitations in the state of the art and will extend the state of the art to improve the enhancements for the effective and efficient performance of the big data stores on distributed systems. The taxonomy not only highlights the design, the similarities, and the differences within state of the art for different types of data partitioning algorithms but also identifies the areas that need further research.