Safe Hardware Access with the Xen Virtual Machine Monitor
Keir Fraser, Steven Hand, Rolf Neugebauer∗, Ian Pratt, Andrew Wareld, Mark Williamson∗
University of Cambridge Computer Laboratory, J J Thomson Avenue, Cambridge, UK
{rstname.lastname}@cl.cam.ac.uk
Abstract
The Xen virtual machine monitor allows multiple operating
systems to execute concurrently on commodity x86 hard-
ware, providing a solution for server consolidation and util-
ity computing. In our initial design, Xen itself contained
device-driver code and provided safe shared virtual device
access. In this paper we present our new Safe Hardware
Interface, an isolation architecture used within the latest
release of Xen which allows unmodified device drivers to
be shared across isolated operating system instances, while
protecting individual OSs, and the system as a whole, from
driver failure.
1 Introduction
We have recently developed Xen [1], an x86-based virtual
machine manager specifically targeting two utility-based
computing environments:
1. organizational compute/data centers, where a large
cluster of physical machines may be shared across dif-
ferent administrative units; and
2. global-scale compute utilities, such as our own
XenoServers [2] initiative, in which completely un-
related customers may lease a set of distributed re-
sources to deploy their own services.
In both of these situations, Xen must provide reliable exe-
cution of OS instances, hard isolation, and accounting and
management for the underlying physical resources.
This paper focuses on our work addressing dependability
on commodity hardware by isolating device driver code yet
retaining the ability to share devices across OS instances.
Device drivers are well known to be a major source of bugs
and system failures, and the sharing of devices raises the
stakes of driver dependability drastically. The wide variety
of hardware available for PCs and the desire to share device
access has led us to an architecture in which we execute
unmodified device drivers in isolated “driver domains”: in
essence, driver-specific virtual machines.
∗Intel Research Cambridge, UK
To achieve this robustly, we have developed a Safe Hard-
ware Interface which allows the containment of practi-
cally all driver failures by limiting the driver’s access to
the specific hardware resources (memory, interrupts, and
I/O ports) necessary for its operation. Our model, which
places device drivers in separate virtual OS instances, pro-
vides two principal benefits: First, drivers are isolated from
the rest of the system; they may crash or be intentionally
restarted with minimal impact on running OSes. Second, a
unified interface to each driver means that drivers may be
safely shared across many OSes at once, and that only a
single low-level driver implementation is required to sup-
port different paravirtualized operating systems.
While general approaches to partitioning and protecting
systems for reliability have been explored over the past
thirty years, they often depend on specific hardware sup-
port [3], or are unconcerned with enterprise-class perfor-
mance and dependability [4, 5]. Our work addresses the
problem of ensuring the reliability of shared device drivers
for enterprise services on the PC architecture without re-
quiring specialized hardware support.
2 Related Work
The current PC I/O architecture presents a multifaceted set
of challenging problems. This section attempts to summa-
rize the great breadth of previous work that has attempted
to tackle individual aspects of the problem. We have drawn
on many of these efforts in our own research. There are two
broad classes of work related to our own. First is a large set
of efforts both in systems software and hardware develop-
ment toward safe isolation. Second are attempts to better
structure the interfaces between devices and their software,
and the OSs and applications they interact with.
2.1 Safe Isolation
Researchers have long been concerned with the inclusion
of extension code in operating systems. Extensible oper-
ating systems [6, 7] explored a broad range of approaches

to support the incorporation of foreign, possibly untrusted
code in an existing OS. Swift et al [4] leverage the experi-
ences of extensibility, particularly that of interposition, to
improve the reliability of Linux device drivers. While their
work claims an improvement in system reliability it demon-
strates the risk of a narrow focus: their approach sacrifices
performance drastically in an attempt to add dependabil-
ity without modifying the existing OS. By addressing the
larger architectural problem and not fixating on a single
OS instance, we provide higher performance and solve a
broader set of issues, while still remaining compatible with
existing systems.
Our implementation, presented in Section 4, uses a virtu-
alization layer to achieve isolation between drivers and the
OS (or OSs) that use them. Providing a low-level systems
layer that is principally responsible for managing devices
was initially explored in Nemesis [8] and the Exokernel [9].
Our work refines these approaches by applying them to ex-
isting systems. Additionally, Whitaker et al [10] speculate
as to the potential uses of a virtualized approach to system
composition, drawing strongly on early microkernel efforts
in Mach [11] among others [12, 13]. Our work represents a
realization of these ideas, demonstrating that isolation can
be provided with a surprisingly low performance overhead.
Commercial offerings for virtualization, such as VMware
ESX Server [14], allow separate OSs to share devices.
While we have previously demonstrated [1] that our ap-
proach to virtualization provides higher performance, this
work moves to focus specifically on additional concerns
such as driver dependability; our implementation is now
not only faster but also accommodates a strictly higher level
of driver dependability.
Several research efforts have investigated hardware-
assisted approaches to providing isolation on the PC plat-
form. The Recovery-Oriented Computing [15] project,
whose goals are similar to our own, have used hardware
for system diagnostics [16], but defer to ‘standard mecha-
nisms’ for isolation. Intel’s SoftSDV [17], which is a de-
velopment environment for operating systems supporting
the IA-64 instruction set, uses PCI riser cards to proxy I/O
requests. While their concern is in mapping device inter-
rupts and DMA into the simulated 64-bit environment, the
same approach could be used to provide device isolation.
Intel has also announced that their new LaGrande architec-
ture [18] will protect memory from device DMA.
2.2 Better Interfaces
Our goal of providing more rigid OS–device interfaces
is hardly new. Most notably, corporate efforts such as
UDI [19] have attempted to do just this. There are two key
limitations of UDI that we directly address. Firstly, we en-
force isolation whereas UDI-compliant drivers still execute
in the same protection domain as the operating system, and
thus there is no mitigation of the risks posed by erroneous
Hardware
SoftwareSafe Hardware Interface
Device M
anager
Device
Driver
Device
Driver
Device
Driver
DeviceDevice Device
I/O Space A I/O Space B I/O Space C
Operating
System A
(e.g. Windows XP)
Operating
System B
(e.g. Linux)
Device Channels:
Safe and efficient
interdomain messaging
Unified Interfaces:
Single driver source
Drivers may be shared
Legacy Interfaces:
Original OS driver
Still isolated for safety
No driver sharing or restart
I/O Spaces:
Provide strong vertical
isolation for driver/device
instances
Figure 1: Design of the safe hardware model.
drivers. Secondly, our external perspective avoids the trap
to which vendor consortiums such as UDI often fall vic-
tim: that of ‘interface unioning’. Rather than providing the
aggregate interface present in all existing drivers, we set-
tle on a narrower, idealized interface. While we provide
mechanisms to directly (and safely) expose the hardware
should our interface be too constrictive, we have not found
this to be a problem in our experiences with a large number
of network and storage devices—most relevant for server-
class machines—and several OSs.
Novel OS architectures have long struggled with a lack of
device driver support. The vast number of available de-
vices has compounded this problem, making the adoption
of an existing driver interface attractive for fledgling sys-
tems. Microkernel systems such as Fluke [20] and L4 [21]
have investigated wrapping Linux device drivers in cus-
tomized interfaces [22, 23]. Although the structure of our
architecture is not entirely unlike a microkernel, our intent
is to solve the driver interface issue for all operating sys-
tems on the PC architecture, rather than make a small set
of existing drivers work for a single experimental OS.
Perhaps most closely related to our work are recent at-
tempts by researchers to use a microkernel approach to al-
low the reuse of unmodified device drivers within newer
operating systems [24, 25]. We are less concerned with us-
ing legacy drivers in modern operating systems than with
providing shared access to isolated device drivers.
3 Design
This section presents the high-level design issues that have
driven our work. We begin by discussing the issues in-
volved in achieving isolation between virtualized OS in-
stances, device driver code, and physical devices. In the
second half of this section, we go on to discuss design con-
cerns for unified interfaces.
As illustrated by Figure 1, our architecture comprises three
parts. Firstly, we introduce I/O Spaces which arrange that
devices perform their work in isolation from the rest of the