GCNScheduler: Scheduling Distributed Computing Applications using Graph Convolutional Networks

Abstract

We provide a highly-efficient solution to the classical problem of scheduling task graphs corresponding to complex applications on distributed computing systems. A number of heuristics have been previously proposed to optimize task scheduling with respect to different metrics (e.g. makespan and throughput). However, they tend to be slow to run, particularly for larger problem instances, limiting their applicability in more dynamic systems. Motivated by the goal of solving these problems more rapidly, we propose, for the first time, a graph convolutional network-based scheduler (GCNScheduler). By carefully integrating the inter-task data dependency structure and the computational network into a single input graph, the GCNScheduler can efficiently schedule tasks of complex applications for a given objective. We use simulations to illustrate that not only can our scheme quickly and efficiently learn from existing scheduling schemes, but also it can easily be applied to large-scale settings that current scheduling schemes fail to handle. We demonstrate the generalization of GCNScheduler to unseen real-world applications and show that it achieves almost the same makespan and throughput as benchmarks, while providing several orders of magnitude faster scheduling times.