A stable online scheduling strategy for real-time stream computing over fluctuating big data streams

Abstract

The issue of high system stability is one of the major obstacles for real-time computing over fluctuating big data streams. A stable scheduling is more important than an efficient scheduling for stream applications, especially when a scheduling is to be rescheduled dynamically at runtime. In this paper, a stable online scheduling strategy with makespan guarantee SOMG is discussed, which includes the following features: 1) profiling mathematical relationships between system stability, response time, and resource utilization, and indicating conditions to meet the high system stability and acceptable response time objectives; 2) optimizing the structure of a data stream graph by quantifying and adjusting vertices of the graph; and 3) scheduling a data stream graph with heuristic critical path scheduling mechanism, which is subject to response time constraints, rescheduling only key vertices on dynamically changing critical path of the graph, and considering the historical information of current scheduling to maximize system stability with response time aware. Experimental results conclusively demonstrate that the SOMG framework has higher potential of providing enhancement on efficient system stability and guaranteeing significant response time. It efficiently and effectively makes a tradeoff between high system stability and acceptable response time objectives in big data stream computing environments.