Simulation of MapReduce across geographically distributed datacentres using CloudSim

Abstract

Analysis of geo-distributed Big Data has been recently gaining importance. This is addressed either by copying data to a single data centre, or by processing data locally at each datacentre and aggregating the outputs at a single datacentre. Both involve expensive data transfers over wide area networks (WAN). In this work, we analyzed different models proposed for distributed MapReduce in various papers and selected a feasible model to simulate Map Reduce across distributed data centers. We have designed an extension to CloudSim and CloudSimEx to support three methods of implementing geo-distributed MapReduce. A heuristic decision algorithm is devised based on input, intermediate, and output files sizes to select suitable execution path.