Holmes: SMT Interference Diagnosis and CPU Scheduling for Job Co-location

Abstract

Co-location of latency-critical services with best-effort batch jobs is commonly adopted in production systems to increase resource utilization. Although memory and CPU isolation have been extensively studied, we find Simultaneous Multi-Threading (SMT) technology imposes non-trivial interference on memory access which jeopardizes efficient co-location and performance assurance of latency-critical services. However, there is not an existing metric to quantitatively measure and lacks a deterministic approach to tackle SMT interference on memory access. We present Holmes, a user-space approach to SMT interference diagnosis and adaptive CPU scheduling for efficient job co-location in multi-tenant systems. Holmes tackles two challenges: accurately measuring SMT interference on memory access, and efficiently adjusting CPU allocation to achieve low latency and high resource utilization at the same time. It leverages CPU hardware performance events to diagnose SMT interference on memory access and form a metric. It deploys an interference-aware scheduler to adaptively allocate CPU cores to latency-critical services and batch jobs. Experiments with four real-world key-value stores show that compared to a representative CPU isolation approach, Holmes reduces the average (99th percentile) query latency by up to 49.0% (52.3%) for four real-world latency-critical services. It also significantly improves convergence speed, resource utilization, and system throughput.