USENIX Association
Proceedings of the FREENIX Track:
2004 USENIX Annual Technical Conference
Boston, MA, USA
June 27–July 2, 2004
© 2004 by The USENIX Association All Rights Reserved For more information about the USENIX Association:
Phone: 1 510 528 8649 FAX: 1 510 548 5738 Email: office@usenix.org WWW: http://www.usenix.org
Rights to individual papers remain with the author or the author's employer.
Permission is granted for noncommercial reproduction of the work for educational or research purposes.
This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein.

Xen and the Art of Repeated Research

Bryan Clark, Todd Deshane, Eli Dow, Stephen Evanchik, Matthew Finlayson, Jason Herne,
Jeanna Neefe Matthews
Clarkson University
{clarkbw, deshantm, dowem, evanchsa, finlayms, hernejj, jnm}@clarkson.edu

Abstract

Xen is an x86 virtual machine monitor produced by the University of Cambridge Computer Laboratory and released
under the GNU General Public License. Performance results comparing XenoLinux (Linux running in a Xen virtual
machine) to native Linux as well as to other virtualization tools such as User Mode Linux (UML) were recently pub-
lished in the paper “Xen and the Art of Virtualization” at the Symposium on Operating Systems Principles (October
2003). In this study, we repeat this performance analysis of Xen. We also extend the analysis in several ways, includ-
ing comparing XenoLinux on x86 to an IBM zServer. We use this study as an example of repeated research. We
argue that this model of research, which is enabled by open source software, is an important step in transferring the
results of computer science research into production environments.

1. Introduction
Repeated research is a well-respected model of investi-
gation in many sciences. Independent tests of published
research are valued because they document the general
applicability of results. In addition, repeated research
often sheds new light on aspects of a work not fully
explored in the original publication.

In computer science, however, it is most common for
researchers to report results from testing the software
that they themselves have implemented. There are many
reasons for this, including the wide variety of hardware
and software platforms and the difficulty transferring
fragile research software to a new environment. How-
ever, without independent trials, it is difficult to estab-
lish reported experience as repeatable fact.

Computer systems researchers often note with dismay
the number of great ideas that are not incorporated into
production computer systems. We argue that encourag-
ing repeated research is an important step towards this
transfer of technology. Researchers who release their
code to the open source community make a valuable
step towards encouraging repeated research in computer
science.

In this paper, we present results that repeat and extend
experiments described in the paper “Xen and Art of
Virtualization” by Barham et al. from SOSP-03.
[Xen03]. Xen is an x86 virtual machine monitor pro-
duced by the University of Cambridge Computer Labo-
ratory in conjunction with Microsoft Research and Intel
Research. Xen has been released under the GNU Gen-
eral Public License at xen.sourceforge.net.

In [Xen03], Barham et al. explore the performance of
XenoLinux – Linux running in Xen. They compare per-
formance to native Linux as well as to other virtualiza-
tion tools such as User Mode Linux (UML) and
VMWare Workstation. They also examine how the
performance of Xen scales as additional guest operating
systems are created.

In this paper, we first report the results of repeating
measurements of native Linux, Xenolinux and User
Mode Linux on hardware almost identical to that used
in the Xen paper. Second, we present results comparing
Xen to native Linux on a less powerful PC. Third, we
evaluate Xen as a platform for virtual web hosting.
Fourth, we compare the performance of benchmarks
running in XenoLinux to the same benchmarks running
in Linux on an IBM zServer that we won as a prize in
the 2001 IBM Linux Scholar Challenge competition.
Finally, we discuss our general experiences with re-
peated research.

We structure the rest of our paper around a set of ques-
tions and their answers.

• Can we reproduce the results from the SOSP-
03 Xen paper?
• Could we realistically use Xen for virtual web
hosting?
• Do you need a $2500 Dell Xeon Server to run
Xen effectively, or will a 3 year old x86 do the
job?