Scheduling and capacity design in overlay computing systems

Abstract

Parallel to new developments in the fields of computer networks and high performance computing, effective distributed systems have emerged to answer the growing demand to process huge amounts of data. Comparing to traditional network systems aimed mostly to send data, distributed computing systems are also focused on data processing what introduces additionally requirements in the system performance and operation. In this paper we assume that the distributed system works in an overlay mode, which enables fast, cost-effective and flexible deployment comparing to traditional network model. The objective of the design problem is to optimize task scheduling and network capacity in order to minimize the operational cost and to realize all computational projects assigned to the system. The optimization problem is formulated in the form of an ILP (Integer Linear Programming) model. Due to the problem complexity, four heuristics are proposed including evolutionary algorithms and Tabu Search algorithm. All methods are evaluated in comparison to optimal results yielded by the CPLEX solver. The best performance is obtained for the Tabu Search method that provides average results only 0.38% worse than optimal ones. Moreover, for larger problem instances with 20-minute limit of the execution time, the Tabu Search algorithm outperforms CPLEX for some cases. © 2012 Springer-Verlag.