536 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 35, NO. 4, JULY 2005
Evaluation of Virtual Fixtures for a Robot
Programming by Demonstration Interface
Jacopo Aleotti, Student Member, IEEE, Stefano Caselli, Member, IEEE, and Monica Reggiani, Member, IEEE
Abstract—We investigate the effectiveness of several types of vir-
tual fixtures in a robot programming by demonstration interface.
We show that while all types of virtual fixtures examined yield a
significant reduction in the number of errors in tight tolerance
peg-in-hole tasks, color and sound fixtures generally outperform
a tactile fixture in terms of both execution time of successful trials
and error rate. We have found also that when users perceive that
the task is very difficult but the system is providing some help by
means of a virtual fixture, they tend to spend more time trying to
achieve a successful task execution. Thus, for difficult tasks the
benefits of virtual fixturing are better reflected in a reduction of
the error rate than in a decreased execution time. We conjecture
that these trends are related to the limitations of currently avail-
able interfaces for human-robot interaction through virtual envi-
ronments and to the different strategies adopted by the users to
cope with such limitations in high-accuracy tasks.
Index Terms—Human factors, robot programming, user inter-
faces, virtual reality (VR).
I. INTRODUCTION
ONE OF THE major challenges for efficient programmingof complex robot tasks is to achieve, in the context of
human-robot interaction, programming by demonstration (PbD)
capabilities. The aim of robot PbD is to simplify the problem
of describing robot actions. Traditional robotic applications re-
quire complex interfaces that are unsuitable for standard desktop
environments and untrained users. PbD offers a way to build
user-friendly interfaces that overcome several programming dif-
ficulties. For manipulation tasks, a PbD system provides the user
with a conceptually simple way to instruct a robotic platform just
by showing with his/her own hands how to do a particular task.
PbD systems can be classified into two main categories de-
pending on the way demonstration is carried out. The most gen-
eral way is performing the demonstration in the real environ-
ment [1]–[5]. This approach requires difficult task recognition
and segmentation capabilities. Another approach involves per-
forming the demonstration in a simulated environment where
tracking user’s actions is easier [6]–[10]. A major difficulty of
this approach is that in cluttered virtual environments, such as
for robotic assemblies, it requires that fine manipulation tasks
can be tracked with high fidelity.
Manuscript received August 2, 2004; revised February 15, 2005 and
March 14, 2005. This paper was recommended by Associate Editors
J. A. Adams and M. Skubic. This work was supported by the Italian Ministry
of Education, University and Research (MIUR) under project RoboCare
(a multiagent system with intelligent fixed and mobile robotic components).
The authors are with the Dipartimento di Ingegneria dell’Informazione, Uni-
versity of Parma, Parco Area delle Scienze, 181/A, 43100 Parma, Italy (e-mail:
aleotti@ce.unipr.it; caselli@ce.unipr.it; reggiani@ce.unipr.it).
Digital Object Identifier 10.1109/TSMCA.2005.850604
In PbD in simulated environments, human-robot interaction
is essentially mediated by the virtual environment. It involves
user-interfaces design, human factors, performance evaluation
and cognitive psychology. Most traditional human-computer in-
teraction systems focus on the design of applications that convey
to the user only graphical information through a visual channel.
To enhance the perception of advanced data sets such as a vir-
tual environment, a graphical interface can be augmented with
additional sensory aids such as auditory and haptic feedbacks
[11], [12].
Synthetic aids integrated into a virtual environment are called
virtual fixtures. In his seminal work, Rosenberg [13] defines vir-
tual fixtures as “abstract sensory information overlaid on top of
reflected sensory feedback from a remote environment.” These
fixtures can be composed of “haptic, visual, auditory, and tactile
sensations, used alone or in cross-modal combinations” [13].
Virtual fixtures have been later extended from telerobotic ma-
nipulation to virtual and simulation environments in a number
of systems [7], [12], [14], [15], [16]. Virtual fixtures are thus
perceptual overlays designed to reduce the physical and psycho-
logical demands that arise during the execution of a task. Vir-
tual fixtures have been proven to improve the operator’s perfor-
mance in terms of both speed and accuracy, since they can alert
the user for changes in the state of the environment and support
hand-eye coordination for object manipulation tasks [13], [17].
In this paper, we address the problem of evaluating a set of
virtual fixtures (implemented using visual, auditory, and tactile
sensory feedback) in an interactive virtual environment for ma-
nipulation tasks. The virtual environment investigated in the ex-
periments is the demonstration interface of a PbD system for
robotic assembly tasks. Past studies have found that the quan-
titative impact of virtual fixtures is task dependent [13], [14],
[18]. Thus, the objective of this work is to find whether there is
a dominating artificial fixture that can be simply introduced in
virtual environments to improve the effectiveness of user inter-
action. We focus on systems comprising a virtual reality (VR)
glove, since such a device is commonly available in interactive
environments for manipulation tasks. Hence, a secondary goal
is to determine to what extent a VR glove is suitable for high
precision manipulation tasks.
In [10], we investigated the benefits of the vibrotactile feed-
back available in a VR glove in a few, relatively simple manip-
ulation tasks which can occur in a PbD interface exploiting a
virtual environment. The results demonstrated that this type of
haptic feedback decreases the average demonstration times. In
this paper, we focus on a specific peg-in-hole task at higher diffi-
culties, introducing additional virtual fixtures such as visual and
auditory aids, both individually and in multimodal combination.
1083-4427/$20.00 © 2005 IEEE