Enhhanced and efficient memory model for hadoop mapreduce

Abstract

Usage of high-performance computing (HPC) infrastructure adopting cloud-computing environment offers an efficient solution for executing data intensive application. MapReduce (MR) is the favored high performance parallel computing framework used in BigData study, scientific, and data intensive applications. Hadoop is one of the significantly used MR based parallel computing framework by various organization as it is freely available open source framework from Apache foundation. The existing Hadoop MapReduce (HMR) based makespan model incurs memory and I/O overhead. Thus, affecting makespan performance. For overcoming research issues and challenges, this manuscript presented an efficient parallel HMR (PHMR) makespan model. The PHMR includes a parallel execution scheme in virtual computing worker to reduce makespan times using cloud computing framework. The PHMR model provides efficient memory management design within the virtual computing workers to minimize memory allocation and transmission overheads. For evaluating performance of PHMR of over existing model experiment are conducted on public cloud environment using Azure HDInsight cloud platform. Different application such as bioinformatics, tex mining, stream, and non-stream application is considered. The overall result obtained shows superior performance is attained by PHMR over existing model in term of makespan time reduction and correlation among practical and theoretical makespan values.