Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus

Abstract

The latest trend in high performance computing and big data applications inspires new solutions of in-memory filesystems to achieve high throughput of file data accesses. However, new concerns arise because intensive in-memory file accesses can cause contention problem on memory bus. With existing schedulers, such as Complete Fair Scheduler (CFS) in Linux system, memory access intensive processes can excessively occupy memory bandwidth. As a result, the performance of a set of running processes is degraded. In this paper, we propose a scheduling algorithm to consider memory bandwidth consumption during scheduling and improve the completion time of a given working set. We first present the data to show that CFS can not efficiently handle a set of running processes involving intensive in-memory file accesses. Then, a heuristic algorithm is proposed to solve the scheduling problem with bandwidth consumption and execution time consideration. The scheduling algorithm is implemented and extensively experimented in Linux system. The experimental results show that the completion time is reduced up to 33.3% compared with that generated by Linux scheduler CFS.