Queuing-oriented job optimizing scheduling in cloud mapreduce

Abstract

Cloud MapReduce, as an implementation of MapReduce framework on Cloud for big data analysis, is facing the unknown job makespan and long wait time problem, which have seriously affected the service quality. The Inefficient virtual machine allocation is one critical causing factor. Based on the M/M/1 model, a new queuing equation is built to ensure the virtual machine with the high efficiency. By jointing queuing equation and objectives function, a two variables equation group is designed to compute the desired virtual machine number for different jobs. According to the desired virtual machine number of each job, we developed a queuing-oriented job optimizing scheduling algorithm, called QTJS, to optimal job scheduling and enhance the resource utilization in Cloud MapReduce. Extensive experiments show that our QTJS algorithm consumes less job execution time and performs better efficiency than other three algorithms.