Partitioning in Apache Spark

Abstract

Apache Spark performs in-memory computation. The data structure used is Resilient Distributed Datasets (RDDs). These RDDs are partitioned using inbuilt Hash and Range Partitioning. We propose a partition scheme which uses modular division on keys of elements with numbers from 2 to 10. This scheme works on smaller datasets in order to enhance the execution time.