Facilitating the Development of Soft Devices
Andrew Warfield, Steven Hand, Keir Fraser and Tim Deegan
University of Cambridge Computer Laboratory, J J Thomson Avenue, Cambridge, UK
{firstname.lastname}@cl.cam.ac.uk
1 Introduction
Device-level interfaces in operating systems present a very
useful cut-point for researchers to experiment with new
ideas. By virtualizing these interfaces, developers can cre-
ate soft devices, which are used in the same way as nor-
mal hardware devices, but provide extra functionality in
software. Recent years have shown this approach to be of
considerable interest: a few examples of block device ex-
tension include the addition of intrusion detection systems
to disk interfaces [1], the development of “semantically
smart” disks [2], and that of time-travel block devices [3].
Other devices, such as network interfaces, have similarly
been extended.
Working at the device interface allows an examination of
the functional separation between hardware and software:
researchers can simulate new features as if they were prop-
erties of the device itself. As simple examples, block or
network device interfaces might be extended to compress
or encrypt data before it is written to disk or transmitted.
Alternatively, it may be desirable to prototype entirely new
devices in software, bound to existing interfaces, for in-
stance a content-addressable disk.
Unfortunately, researchers face a challenge in extending
devices in this manner. Implementors must typically mod-
ify an existing operating system to add the new function-
ality, often by creating OS-specific pseudo-devices. This
requirement means learning the OS source and writing
scaffolding code to intercept events. Moreover, where
new functionality must be developed in-kernel it is diffi-
cult to debug and crashes are not contained. Finally, these
low-level developments are difficult to share and maintain
across systems, as they will be specific to the OS, or even
specific version thereof, that it has been developed within.
This paper presents a solution to the problems associated
with developing soft devices by extending the existing de-
vice interface in Xen [4]. Xen is a virtual machine monitor
(VMM) for the IA32 architecture that paravirtualizes hard-
ware: Rather than attempting to present a fully virtualized
hardware interface to each OS in a Xen environment, guest
OSes are modified to use a simple, narrow and idealized
view of hardware. Soft devices take advantage of these
narrow interface to capture and transform block requests,
network packets, and USB messages.
As an initial example of this approach, we have imple-
mented a block tap, which is an interface to facilitate the
development of soft devices for block device access. The
block tap allows soft devices to be constructed as user-
space applications in an entirely isolated virtual machine.
This strong isolation from the remainder of the system al-
lows a single soft device to work with any OS and hardware
available on Xen, and allows developers to work with high-
level languages and debuggers. While our approach aims
to facilitate development it still provides a high level of per-
formance, sustaining 50MB/s read throughput for disk re-
quests in our experiments.
2 Device Access in Xen
The existing approach to device access in Xen makes use
of split device drivers, and has been described in detail in
[5]. Figure 1 illustrates a split driver: A device driver VM
is granted specific access to the physical hardware that it
will manage. This VM runs an existing, unmodified (e.g.
Linux) device driver to access the device. In addition it
runs a back-end driver, which provides a simple narrow in-
terface to the device. Operating systems wishing to access
the device will use a front-end driver, and interact with the
back-end over a device channel, which is a shared-memory
communications primitive.
This approach aims to improve system stability while still
supporting existing device drivers by isolating drivers in
a single VM, away from both other OSes and the VMM.
Perhaps more importantly though, the split driver interface
simplifies device access for operating systems above Xen,
as an OS need only implement a single front-end driver to

Xen
Device VM Guest OS
Block
Front-end
Driver
Block
Back-end
Driver
Physical
Device
Driver
device
channel
Split Device Drivers in Xen
Physical driver runs in an isolated VM, connected over a shared memory device
channel to a guest VM accessing the device.
Figure 1: Split device drivers in Xen
support an entire class of devices (e.g. block storage).
Xen’s current split block device illustrates exactly how nar-
row this interface is. The device channel shown in Figure 1
is a single page of memory shared between the two VMs.
A bi-directional ring buffer is used to pass messages. Pages
of data are attached and mapped separately. The interface
has only three commands: READ, WRITE, and PROBE.
READ and WRITE provide block-level access to data, while
PROBE returns a list of accessible block devices.
3 Implementing a soft device Interface
This section describes how we have extended Xen’s exist-
ing split device interface to support the development of soft
devices. We describe an implementation of a block tap that
allows the construction block soft devices. In designing
and implementing the block tap, we have attempted to meet
three general requirements:
1. Make device implementation easy. Our principal
goal is to facilitate the development and exploration
of new functionality for device interfaces. We have
chosen to give developers the option of receiving de-
vice requests through a user-level interface in the soft
device domain, allowing development in a safe envi-
ronment with a variety of languages and tools.
2. Do not modify the existing guest OS or device VM.
While Xen itself achieves performance by modifying
the guest operating system, we desire that the soft de-
vice interface leave the attached front-end and back-
end VMs unmodified. This approach allows the devel-
opment of soft devices between any hardware and OS
combination supported by Xen without necessitating
the modification (or even understanding) of that code.
Additionally, the soft device interface may be used to
trace, debug, or modify an existing split device con-
nection. As such, all soft device implementation exists
within an isolated VM using only the shared-memory
device channels as an interface.
3. Maintain satisfactory performance. We hope to use
soft devices in practical situations, and not just as a
Xen
Block Tap VM
Soft Device Application
device
channel
device
channel
user dev
channel
user dev
channel
Block Tap Device Structure
back-end
ring page
front-end
ring page
additional address space
for data page mappings
to back-end
in Device VM
to front-end
in Guest VM
Message
Switch
/dev/blocktap is mapped into application memory:
The application accesses the block device interface to generate and receive block
messages. The message switch has a variety of forwarding modes.
Figure 2: block tap structure.
prototyping tool. By taking advantage of the perfor-
mance allowed through request batching, we hope to
maintain a high level of throughput for soft devices.
The remainder of this section describes our block tap im-
plementation addressing these requirements. The block tap
is currently about 1300 lines of commented C code, and
runs in a Linux-based VM.
3.1 A Switch for Block Requests
The potential ways in which a soft device interface might
be used are varied. Developers may desire to simply trace
request traffic in order to monitor usage patterns, they may
wish to modify in-flight requests, or they may desire to
construct a terminating device, which does not forward re-
quests at all.
In order to accommodate these varied modes of operation,
we have implemented the soft device interface as a request
switch. The driver is plumbed into the device channel be-
tween the front-end and back-end domains. In this position,
all block requests pass through it.
The new driver acts as a switch, shown in Figure 2, for-
warding messages across four rings. Two of these rings are
inter-VM shared memory rings as described above. They
connect to the front-end and back-end drivers that the soft
device interface has been placed between. Two additional
rings extend from the block tap up to application space in
the same VM. These rings are accessed through a character
device that can be mapped by applications.
An ioctl() to this character device is used to set the
switching mode used for block messages. Three common
modes are described here, and shown in Figure 3.
MODE PASSTHROUGH is the lowest-overhead switching
configuration. In this mode, messages are passed straight
through the driver on to the opposite ring, and completely