A comparative study of real workload traces and synthetic workload models for parallel job scheduling

Abstract

Two basic approaches are taken when modeling workloads in simulation-based performance evaluation of parallel job scheduling algorithms: (1) a carefully reconstructed trace from a real supercomputer can provide a very realistic job stream, or (2) a flexible synthetic model that attempts to capture the behavior of observed workloads can be devised. Both approaches require that accurate statistical observations be made and that the researcher be aware of the applicability of a given trace for his or her experimental goals. In this paper, we compare a number of real workload traces and synthetic workload models currently used to evaluate job scheduling and allocation strategies. Our results indicate that the choice of workload model alone - real workload trace versus synthetic workload models - did not significantly affect the relative performance of the algorithms in this study (two scheduling algorithms and three static processor allocation algorithms). Almost all traces and models gave the same ranking of algorithms from best to worst. However, two specific workload characteristics were found to significantly affect algorithm performance: (a) proportion of powerof- two job sizes and (b) degree of correlation between job size and job runtime. When used in the experimental evaluation of resource management algorithms, workloads differing in these two characteristics may lead to discrepant conclusions.