A Complexity Aware Scheduler with Dynamic Slot Allocation for Cloud Video Transcoding

Abstract

For a multimedia cloud computing platform, it needs to perform video transcoding to provide bandwidth-compatible bit-streams for users. In configuring a MapReduce system, it allocates slots to workers with the assumption of homogeneous worker power and performs task scheduling by assuming equal task time complexity. However, the computing power of a practical cluster of workers and the time complexity of tasks is time-varying in any case. The task-scheduling algorithm has to well manipulate these heterogeneous cloud resources and task complexity. We first find a good partition size for a video transcoding job for efficient processing. We proposed a Complexity-Aware Scheduling (CAS) algorithm that reorders task assignment priorities according to task complexity to maintain load-balancing operations. By utilizing a Neural Network model to refine the task complexity estimation for the CAS(CASNN), the scheduling and transcoding speedup performances can further be improved. Based on the CASNN, we proposed a Dynamically Adjusting Slot number allocation (DAS) method, DASCASNN, to adjust the slots according to resource utilization status to improve the processing performance. Experimental results show the proposed DASCASNN can help reduce 30% of the transcoding time on average as compared to available schedulers and increase the resource utilization rates from 82.7% to 98%.