Two-Stage Optimization-Learning Framework for Uncertainty-Aware Multi-Zonal Data Center Energy Management

Abstract

This paper proposes a hybrid optimization framework that synergizes mixed-integer linear programming (MILP) and reinforcement learning (RL) to minimize operational costs and carbon emissions in multi-zonal data centers integrated with renewable generation and battery storage. The approach addresses the dual challenges of day-ahead planning under forecasted conditions and real-time adaptation to stochastic demand and renewable intermittency. A deterministic MILP model performs advance scheduling with reliability constraints derived from complementary cumulative distribution functions (CCDFs) to ensure robust renewable allocation. Three operational paradigms are evaluated: pure deterministic MILP optimization, pure RL control, and a hybrid framework where RL dynamically adjusts MILP-derived schedules via residual control to accommodate real-time uncertainties. The experimental results demonstrate that, for deterministic conditions, RL achieves performance comparable to MILP in energy utilization, peak demand, and cost. Under demand uncertainty, the hybrid and RL-based approaches reduce operational costs by up to 33% compared to uncertainty-aware MILP execution. These savings stem from significantly lower grid peak demand, achieved by prioritizing renewable energy consumption, strategically dispatching batteries, and resorting to grid power only during extreme demand spikes. The hybrid framework effectively balances optimality and computational efficiency, offering a practical solution for uncertainty-resilient energy management in data centers.