Performance and cost evolution of dynamic increase hadoop workloads of various datacenters

Abstract

In the past years, Datacenters and Clusters data processing are incredibly crucial tasks. To addressing these issues, many researchers have covered up. The MapReduce is an open-source Hadoop structure expected for managing and delivering disseminated vast terabyte information on immense clusters. Its key duty is to reduce the conclusion time for large clusters of MapReduce Jobs. Hadoop Cluster has limited fixed slot design for cluster lifespan. This preset slot configuration may increase the completion time (makespan) and decrease the system resource utilization. The present open Hadoop source permits simply static slot configuration, similarly set of map slots in check to decrease slots all through the cluster lifespan. Such fixed configuration may guide to extend the completion time and the resource utilization of system will decrease. The proposed novel technique for minimizing the makespan of given set using slot-ratio among map and reduce tasks. Through utilizing the workload data of recently finished jobs it allocates slots to map and decrease tasks dynamically.