Predictive Load Balancing in Distributed Systems: A Comparative Study of Round Robin, Weighted Round Robin, and a Machine Learning Approach †

Abstract

Load balancing is a widely adopted strategy in modern distributed systems because it distributes workloads across servers, mitigating overload and improving overall performance. However, the rapid growth of such systems has created a need for more adaptive strategies to ensure optimal utilization and responsiveness of resources. Traditional algorithms such as Round Robin (RR) and Weighted Round Robin (WRR) assign requests without considering server states or request characteristics. We implement a machine learning (ML)–based predictive load balancer, forecasting the latency of a request based on the request itself and container parameters, specifically the average latency of the last 50 requests and the count of active requests, and evaluate it against RR and WRR. For the experiment, synthetic data were generated to replicate real-world requests by creating random URL and method combinations, attaching a task size in Million Instructions (MI), and distributing them among three containers with varying resources according to the load balancing strategies described above. Under the conditions tested, the ML approach achieved the worst performance, trailing both RR and WRR in terms of throughput and average latency, although the model accuracy was sufficiently high (R2 = 0.8+). Post hoc analysis indicates that limited and occasionally stale runtime features caused the load balancer to direct all requests to a single container until the next statistics update, since that container was considered the ‘best’ during that interval.