Fabric: A Platform for Secure Distributed Computation and Storage
Jed Liu Michael D. George K. Vikram
Xin Qi Lucas Waye Andrew C. Myers
{liujed,mdgeorge,kvikram,qixin,lrw,andru}@cs.cornell.edu
Department of Computer Science
Cornell University
Abstract
Fabric is a new system and language for building secure dis-
tributed information systems. It is a decentralized system
that allows heterogeneous network nodes to securely share
both information and computation resources despite mutual
distrust. Its high-level programming language makes distri-
bution and persistence largely transparent to programmers.
Fabric supports data-shipping and function-shipping styles
of computation: both computation and information can move
between nodes to meet security requirements or to improve
performance. Fabric provides a rich, Java-like object model,
but data resources are labeled with confidentiality and in-
tegrity policies that are enforced through a combination of
compile-time and run-time mechanisms. Optimistic, nested
transactions ensure consistency across all objects and nodes.
A peer-to-peer dissemination layer helps to increase avail-
ability and to balance load. Results from applications built
using Fabric suggest that Fabric has a clean, concise pro-
gramming model, offers good performance, and enforces se-
curity.
1 Introduction
We rely on complex, distributed information systems for
many important activities. Government agencies, banks,
hospitals, schools, and many other enterprises use distributed
information systems to manage information and interact with
the public. Current practice does not offer general, princi-
pled techniques for implementing the functionality of these
systems while also satisfying their security and privacy re-
quirements. This lack motivates the creation of Fabric, a
platform for building secure distributed information systems.
It is particularly difficult to build secure federated sys-
tems, which integrate information and computation from in-
dependent administrative domains—each domain has poli-
cies for security and privacy, but does not fully trust other
domains to enforce them. Integrating information from dif-
ferent domains is important because it enables new services
and capabilities.
To illustrate the challenges, consider the scenario of two
medical institutions that want to securely and quickly share
patient information. This goal is important: according to a
1999 Institute of Medicine study, at least 44,000 deaths an-
nually result from medical errors, with incomplete patient
information identified as a leading cause [25]. However, au-
tomated sharing of patient data poses difficulties. First, the
security and privacy policies of the two institutions must be
satisfied (as mandated by HIPAA [22] in the U.S.), restrict-
ing which information can be shared or modified by the two
institutions. Second, a patient record may be updated by both
institutions as treatment progresses, yet the record should be
consistent and up to date when viewed from the two institu-
tions. It is inadequate to simply transmit a copy of the record
in a common format such as XML, because the copy and
the original are likely to diverge over time. Instead, both in-
stitutions should have easy, secure, consistent, and efficient
access to what is logically a single patient record.
Scenarios like this one inspire the development of Fab-
ric, a federated system that supports secure, shared access to
information and computation, despite distrust between co-
operating entities. The goal of Fabric is to make secure dis-
tributed applications much easier to develop, and to enable
the secure integration of information systems controlled by
different organizations.
To achieve this goal, Fabric provides a shared computa-
tional and storage substrate implemented by an essentially
unbounded number of Internet hosts. As with the Web, there
is no notion of an “instance” of Fabric. Two previously non-
interacting sets of Fabric nodes can interact and share in-
formation without prior arrangement. There is no central-
ized control over admission: new nodes, even untrustworthy
nodes, can join the system freely.
Untrustworthy nodes pose a challenge for security. The
guiding principle for security in Fabric is that one’s security
should never depend on components of the system that one
does not trust. Fabric provides security assurance through
a combination of mechanisms at the language and system
levels.
Fabric gives programmers a high-level programming ab-
straction in which security policies and some distributed
computing features are explicitly visible to the programmer.
Programmers access Fabric objects in a uniform way, even
though the objects may be local or remote, persistent or non-
persistent, and object references may cross between Fabric

nodes.
The Fabric programming language is an extension to
the Jif programming language [33, 36], in turn based on
Java [48]; Fabric extends Jif with support for distributed
programming and transactions. Like Jif, Fabric has several
mechanisms, including access control and information flow
control, to prevent untrusted nodes from violating confiden-
tiality and integrity. All objects in Fabric are labeled with
policies from the decentralized label model (DLM) [34],
which expresses security requirements in terms of princi-
pals (e.g., users and organizations). Object labels prevent a
node that is not trusted by a given principal from compromis-
ing the security policies of that principal. Therefore, Fabric
has fine-grained trust management that allows principals to
control to what extent other principals (and nodes) can learn
about or affect their information.
To achieve good performance while enforcing security,
Fabric supports both data shipping, in which data moves to
where computation is happening, and function shipping, in
which computations move to where data resides. Data ship-
ping enables Fabric nodes to compute using cached copies
of remote objects, with good performance when the cache is
populated. Function shipping enables computations to span
multiple nodes. Inconsistency is prevented by performing all
object updates within transactions, which are exposed at the
language level. The availability of information, and scala-
bility of Fabric, are increased by replicating objects within a
peer-to-peer dissemination layer.
Of course, there has been much previous work on mak-
ing distributed systems both easier to build and more se-
cure. Prior mechanisms for remotely executing code, such
as CORBA [40], Java RMI [24], SOAP [50] and web ser-
vices [5], generally offer only limited support for informa-
tion security, consistency, and data shipping. J2EE persis-
tence (EJB) [16] provides a limited form of transparent ac-
cess to persistent objects, but does not address distrust or dis-
tributed computation. Peer-to-peer content-distribution and
wide-area storage systems (e.g., [15, 19, 27, 43]) offer high
data availability, but do little to ensure that data is neither
leaked to nor damaged by untrusted users. Nor do they en-
sure consistency of mutable data. Prior distributed systems
that enforce confidentiality and integrity in the presence of
distrusted nodes (e.g., [55, 11, 54]) have not supported con-
sistent computations over persistent data.
Fabric integrates many ideas from prior work, including
compile-time and run-time information flow, access control,
peer-to-peer replication, and optimistic transactions. This
novel integration makes possible a higher-level program-
ming model that simplifies reasoning about security and con-
sistency. Indeed, it does not seem possible to provide a high-
level programming model like that of Fabric by simply lay-
ering previous distributed systems abstractions. Several new
ideas were also needed to make Fabric possible:
• A programming language that integrates information
flow, persistence, transactions, and distributed compu-
tation.
• A trust ordering on information-flow labels, support-
ing reasoning about information flow in distributed sys-
tems.
• An integration of function shipping and data shipping
that also enforces secure information flows within and
among network nodes.
• A way to manage transactions distributed among mutu-
ally distrusting nodes, and to propagate object updates
while enforcing confidentiality and integrity.
Fabric does not require application developers to aban-
don other standards and methodologies; it seems feasible
for Fabric to interoperate with other standards. In fact, Fab-
ric already interoperates with existing Java application code.
It seems feasible to implement many existing abstractions
(e.g., web services) using Fabric. Conversely, it seems fea-
sible to implement Fabric nodes by encapsulating other ser-
vices such as databases. We leave further work on interoper-
ability to the future.
The rest of this paper describes the design and implemen-
tation of Fabric. Section 2 presents the Fabric architecture in
more detail. Section 3 introduces the Fabric language. Sec-
tion 4 covers cache and transaction management. Section 5
explains howmultinode transactions are implemented. Some
details of the Fabric implementation are given in Section 6;
with the exception of certain explicitly identified features,
the design described in this paper has been implemented in a
prototype. Section 7 reports on our evaluation of this imple-
mentation, including results for the expressiveness of Fabric
and the performance of a substantial application built using
Fabric. Related work is covered in Section 8, and Section 9
concludes.
2 Architecture
Fabric nodes take on one of the three roles depicted in Fig-
ure 1:
• Storage nodes (or stores) store objects persistently and
provide object data when requested.
• Worker nodes perform computation, using both their
own objects and possibly copies of objects from stor-
age nodes or other worker nodes.
• Dissemination nodes provide copies of objects, giving
worker nodes lower-latency access and offloading work
from storage nodes.
Although Fabric nodes serve these three distinct roles, a
single host machine can have multiple Fabric nodes on it,
typically colocated in the same Java VM. For example, a
store can have a colocated worker, allowing the store to in-
voke code at the worker with low overhead. This capabil-
ity is useful, for example, when a store needs to evaluate a
user-defined access control policy to decide whether an ob-
ject update is allowed. It also gives the colocated worker the
2