Better Never than Late: Meeting Deadlines in Datacenter Networks Christo Wilson Hitesh Ballani Thomas Karagiannis Ant Rowstron christowilson@umail.ucsb.edu hiballan@microsoft.com thomkar@microsoft.com antr@microsoft.com Microsoft Research Cambridge, UK ABSTRACT The soft real-time nature of large scale web applications in today���s datacenters, combined with their distributed work- flow, leads to deadlines being associated with the datacenter application traffic. A network flow is useful, and contributes to application throughput and operator revenue if, and only if, it completes within its deadline. Today���s transport pro- tocols (TCP included), given their Internet origins, are ag- nostic to such flow deadlines. Instead, they strive to share network resources fairly. We show that this can hurt appli- cation performance. Motivated by these observations, and other (previously known) deficiencies of TCP in the datacenter environment, this paper presents the design and implementation of D3, a deadline-aware control protocol that is customized for the datacenter environment. D3 uses explicit rate control to apportion bandwidth according to flow deadlines. Evalua- tion from a 19-node, two-tier datacenter testbed shows that D3, even without any deadline information, easily outper- forms TCP in terms of short flow latency and burst toler- ance. Further, by utilizing deadline information, D3 effec- tively doubles the peak load that the datacenter network can support. Categories and Subject Descriptors: C.2.2 [Computer- Communication Networks]: Network Protocols General Terms: Algorithms, Design, Performance Keywords: Online services, Datacenter, SLA, Deadline, rate control 1. INTRODUCTION The proliferation of datacenters over the past few years has been primarily driven by the rapid emergence of user- facing online services. Web search, retail, advertisement, social networking and recommendation systems represent a few prominent examples of such services. While very different in functionality, these services share a couple of common underlying themes. First is their soft real Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGCOMM���11, August 15���19, 2011, Toronto, Ontario, Canada. Copyright 2011 ACM 978-1-4503-0797-0/11/08 ...$10.00. time nature resulting from the need to serve users in a timely fashion. Consequently, today���s online services have service level agreements (SLAs) baked into their operation [10,16,24]. User requests are to be satisfied within a specified latency target when the time expires, responses, irrespective of their completeness, are shipped out. However, the completeness of the responses directly governs their quality and in turn, operator revenue [17]. Second, the mantra of horizontal scal- ability entails that online services have a partition-aggregate workflow with user requests being partitioned amongst (mul- tiple) layers of workers whose results are then aggregated to form the response [10]. The combination of latency targets for datacenter appli- cations and their distributed workflow has implications for traffic inside the datacenter. Application latency targets cas- cade to targets for workers at each layer targets in the region of 10 to 100ms are common [4], which in turn, yield targets for network communication between the workers. Specifi- cally, for any network flow initiated by these workers, there is an associated deadline. The flow is useful and contributes to the application throughput if, and only if, it completes within the deadline. Today���s datacenter networks, given their Internet origins, are oblivious to any such implications of the application de- sign. Specifically, the congestion control (TCP) and flow scheduling mechanisms (FIFO queuing) used in datacenters are unaware of flow deadlines and hence, strive to optimize network-level metrics: maximize network throughput while achieving fairness. This mismatch can severely impact appli- cation performance this is best illustrated through a couple of simple examples: ��� Case for unfair sharing: Consider two flows that share a bottleneck link one flow has a tighter deadline than the other. As shown in figure 1, with today���s setup, TCP strives for fairness and the flows finish at similar times.1 How- ever, only one flow makes its deadline and is included in the user response. Apart from hurting application performance, this wastes valuable network resources on a non-contributing flow. Alternatively, given explicit information about flow deadlines, the network can distribute bandwidth unequally to meet the deadlines. ��� Case for flow quenching: Consider a common application setting involving multiple servers responding to an aggre- gator simultaneously. The resulting network flows share a bottleneck link and have the same deadline. Further, as- 1We use TCP as a running example since it is used in data- centers today. However, our arguments apply to other TCP variants and proposals like DCTCP [4], XCP [19], etc. 50

d2 d1 Time Flows f1 f2 d2 d1 Time f1 f2 Status Quo Deadline aware X Figure 1: Two flows (f1, f2) with different deadlines (d1, d2). The thickness of a flow line represents the rate allocated to it. Awareness of deadlines can be used to ensure they are met. d Time Status Quo Deadline aware Flows d x x x x x x Time Figure 2: Multiple flows with the same deadline (d). The bottleneck capacity cannot satisfy the deadline if all six flows proceed. Quenching one flow ensures that the others finish before the deadline. sume that congestion on the bottleneck link is such that the aggregate capacity available to these flows is not enough to finish all the flows before the deadline. Figure 2 shows that with today���s setup, all flows will receive their fair share of the bandwidth, finish at similar times and hence, miss the deadline. This, in turn, results in an empty response to the end user. Given flow deadlines, it may be possible to deter- mine that the network is congested and quench some flows to ensure that the remaining flows do meet the deadline. Both these examples reflect a tension between the func- tionality offered by a deadline agnostic network and appli- cation goals. This tension is above and beyond the known deficiencies of TCP in the datacenter environment. These in- clude the incast problem resulting from bursts of flows [8,23], and the queuing and buffer pressure induced by a traffic mix that includes long flows [4]. However, the goal of a deadline-aware datacenter network poses unique challenges: 1. Deadlines are associated with flows, not packets. All packets of a flow need to arrive before the deadline. 2. Deadlines for flows can vary significantly. For example, online services like Bing and Google include flows with a continuum of deadlines (including some that do not have a deadline). Further, datacenters host multiple services with diverse traffic patterns. This, combined with the previous challenge, rules out traditional scheduling solu- tions, such as simple prioritization of flows based on their length and deadlines (EDF scheduling [21]). 3. Most flows are very short ( 50KB) and RTTs minimal (���300��sec). Consequently, reaction time-scales are short, and centralized, or heavy weight mechanisms to reserve bandwidth for flows are impractical. In this paper, we present D3, a Deadline-Driven Delivery control protocol, that addresses the aforementioned chal- lenges. Inspired by proposals to manage network congestion through explicit rate control [11,19], D3 explores the feasibil- ity of exploiting deadline information to control the rate at which endhosts introduce traffic in the network. Specifically, applications expose the flow deadline and size information at flow initiation time. Endhosts use this information to request rates from routers along the data path to the des- tination. Routers thus allocate sending rates to flows to greedily satisfy as many deadlines as possible. Despite the fact that flows get assigned different rates, instead of fair share, D3 does not require routers to maintain per-flow state. By capitalizing on the nature of trust in the datacenter environment, both the state regarding flow sending rates and rate policing are delegated to endhosts. Routers only maintain simple aggregate counters. Further, our design ensures that rate assignments behave like���leases��� instead of reservations, and are thus unaffected by host or router failures. To this effect, this paper makes three main contributions: ��� We present the case for utilizing flow deadline informa- tion to apportion bandwidth in datacenters. ��� We present the design, implementation and evaluation of D3, a congestion control protocol that makes datacenter networks deadline aware. Results from our testbed de- ployment show that D3 can effectively double the peak load that a datacenter can support. ��� We show that apart from being deadline-aware, D3 per- forms well as a congestion control protocol for datacenter networks in its own right. Even without any deadline in- formation, D3 outperforms TCP in supporting the mix of short and long flows observed in datacenter networks. While we are convinced of the benefits of tailoring dat- acenter network design to the soft real time nature of dat- acenter applications, we also realize that the design space for such a deadline-aware network is vast. There exists a large body of work for satisfying application demands in the Internet. While we discuss these proposals in Section 3, the D3 design presented here is heavily shaped by the peculiar- ities of the datacenter environment ��� its challenges (lots of very short flows, tiny RTTs, etc.) and luxuries (trusted environment, limited legacy concerns, etc.). We believe that D3 represents a good first step towards a datacenter network stack that is optimized for application requirements and not a retrofitted Internet design. 2. BACKGROUND: TODAY���S DATACENTERS In this section, we provide a characterization of today���s datacenters, highlighting specific features that influence D3 design. 2.1 Datacenter applications Partition-aggregate. Today���s large-scale, user facing web applications achieve horizontal scalability by partitioning the task of responding to users amongst worker machines (possibly at multiple layers). This partition-aggregate struc- ture is shown in Figure 3, and applies to many web applica- tions like search [4], social networks [6], and recommenda- tion systems [10]. Even data processing services like MapRe- duce [9] and Dryad [18] follow this model. 51