Design and Evaluation of a Continuous Consistency Model
for Replicated Services

Haifeng Yu Amin Vahdat
Computer Science Department
Duke University
Durham, NC 27708fyhf, vahdatg@cs.duke.edu
http://www.cs.duke.edu/ fyhf, vahdatg
Abstract
The tradeoffs between consistency, performance, and
availability are well understood. Traditionally, how-
ever, designers of replicated systems have been forced
to choose from either strong consistency guarantees or
none at all. This paper explores the semantic space be-
tween traditional strong and optimistic consistency mod-
els for replicated services. We argue that an important
class of applications can tolerate relaxed consistency, but
bene t from bounding the maximum rate of inconsistent
access in an application-speci c manner. Thus, we de-
velop a set of metrics, Numerical Error, Order Error,
and Staleness, to capture the consistency spectrum. We
then present the design and implementation of TACT,
a middleware layer that enforces arbitrary consistency
bounds among replicas using these metrics. Finally, we
show that three replicated applications demonstrate sig-
ni cant semantic and performance bene ts from using
our framework.
1 Introduction
Replicating distributed services for increased avail-
ability and performance has been a topic of considerable
interest for many years. Recently however, exponen-
tial increase in access to popular Web services provides
us with concrete examples of the types of services that
would bene t from replication, their requirements and
semantics. One of the primary challenges to replicating
network services is consistency across replicas. Provid-
ing strong consistency (e.g., one-copy serializability [4])
This work is supported in part by the National Science
Foundation (EIA-99772879,ITR-0082912). Vahdat is also sup-
ported by an NSF CAREER award (CCR-9984328). Ad-
ditional information on the TACT project can be found at
http://www.cs.duke.edu/ari/issg/TACT/.
imposes performance overheads and limits system avail-
ability. Thus, a variety of optimistic consistency mod-
els [14, 15, 18, 31, 34] have been proposed for applica-
tions that can tolerate relaxed consistency. Such models
require less communication, resulting in improved per-
formance and availability.
Unfortunately, optimistic models typically provide no
bounds on the inconsistency of the data exported to
client applications and end users. A fundamental ob-
servation behind this work is that there is a continuum
between strong and optimistic consistency that is seman-
tically meaningful for a broad range of network services.
This continuum is parameterized by the maximum dis-
tance between a replica’s local data image and some -
nal image consistent across all replicas after all writes
have been applied everywhere. For strong consistency,
this maximum distance is zero, while for optimistic con-
sistency it is in nite. We explore the semantic space in
between these two extremes. For a given workload, pro-
viding a per-replica consistency bound allows the system
to determine an expected probability, for example, that
a write operation will con ict with a concurrent write
submitted to a remote replica, or that a read operation
observes the results of writes that must later be rolled
back. No such analysis can be performed for optimistic
consistency systems because the maximum level of in-
consistency is unbounded.
The relationship between consistency, availability,
and performance is depicted in Figure 1(a). In moving
from strong consistency to optimistic consistency, ap-
plication performance and availability increases. This
bene t comes at the expense of an increasing probabil-
ity that individual accesses will return inconsistent re-
sults, e.g., stale/dirty reads, or con icting writes. In
our work, we allow applications to bound the maxi-
mum probability/degree of inconsistent access in ex-
change for increased performance and availability. Fig-
ure 1(b) graphs different potential improvements in ap-

Strong
Consistency
Optimistic
Consistency
Performance
Availability
Probability of Inconsistent Access
1 2
0
0
Curve based on
workload/network
characteristics
Performance
P
r
o
b
a
b
i
l
i
t
y
o
f
I
n
c
o
n
s
i
s
t
e
n
t
A
c
c
e
s
s
1 2
(a) (b)
Figure 1: a) The spectrum between strong and optimistic consistency as measured by a bound on the probability of
inconsistent access. b) The tradeoff between consistency, availability, and performance depends upon application
and network characteristics.
plication performance versus the probability of incon-
sistent access, depending on workload/network charac-
teristics. Moving to the right in the gure corresponds
to increased performance, while moving up in the gure
corresponds to increased inconsistency. To achieve in-
creased performance, applications must tolerate a corre-
sponding increase in inconsistent accesses. The tradeoff
between performance and consistency depends upon a
number of factors, including application workload, such
as read/write ratios, probability of simultaneous writes,
etc., and network characteristics such as latency, band-
width, and error rates. At the point labeled 1 in the
consistency spectrum in Figure 1(b), a modest increase
in performance corresponds to a relatively large increase
in inconsistency for application classes corresponding to
the top curve, perhaps making the tradeoff unattractive
for these applications. Conversely, at point 2, large
performance increases are available in exchange for a
relatively small increase in inconsistency for applica-
tions represented by the bottom curve.
Thus, the goals of this work are: i) to explore the is-
sues associated with lling the semantic, performance,
and availability gap between optimistic and strong con-
sistency models, ii) to develop a set of metrics that allow
a broad range of replicated services to conveniently and
quantitatively express their consistency requirements,
iii) to quantify the tradeoff between performance and
consistency for a number of sample applications, and
iv) to show the bene ts of dynamically adapting consis-
tency bounds in response to current network, replica, and
client-request characteristics. To this end, we present
the design, implementation, and evaluation of the TACT
toolkit. TACT is a middleware layer that accepts speci -
cations of application consistency requirements and me-
diates read/write access to an underlying data store. If
an operation does not violate pre-speci ed consistency
requirements, it proceeds locally (without contacting re-
mote replicas). Otherwise, the operation blocks until
TACT is able to synchronize with one or more remote
replicas (i.e., push or pull some subset of local/remote
updates) as determined by system consistency require-
ments.
We propose three metrics, Numerical Error, Order
Error,andStaleness, to bound consistency. Numerical
error limits the total weight of writes that can be applied
across all replicas before being propagated to a given
replica. Order error limits the number of tentative writes
(subject to reordering) that can be outstanding at any one
replica, and staleness places a real-time bound on the de-
lay of write propagation among replicas. Algorithms are
then designed to bound each metric: Numerical error is
bounded using a push approach based solely on local
information; a write commitment algorithm combined
with compulsory write pull enforces order error bound;
and staleness is maintained using real-time vector. To
evaluate the effectiveness of our system, we implement
and deploy across the wide area three applications with
a broad range of dynamically changing consistency re-
quirements using the TACT toolkit: an airline reserva-
tion system, a distributed bulletin board service, and load
distribution front ends to a Web server. Relative to strong
consistency techniques, TACT improves the throughput
of these applications by up to a factor of 10. Relative
to weak consistency approaches, TACT provides strong
semantic guarantees regarding the maximum inconsis-
tency observed by individual read and write operations.
The rest of this paper is organized as follows. Sec-
tion 2 describes the three network services implemented
in the TACT framework to motivate our system archi-
tecture. Section 3 presents the system model and design
we adopt for our target services. Next, Section 4 details
the TACT architecture and Section 5 evaluates the per-
formance of our three applications in the TACT frame-
work. Finally, Section 6 places our work in the context