1

Robot Command, Interrogation and Teaching
via Social Interaction

Peter Ford Dominey, Manuel Alvarez,
Bin Gao, Marc Jeambrun, Ann Cheylus
(dominey@ isc.cnrs.fr),
Institut des Sciences Cognitives, CNRS
67 Blvd. Pinel, 69675 Bron Cedex, France
http://www.isc.cnrs.fr/dom/dommenu-en.htm


Alfredo Weitzenfeld, Adrian Martinez, Antonio Medrano
(alfredo@itam.mx)
ITAM,
San Angel Tizapán, México DF, CP 0100
http://robotica.itam.mx/ingles/index.phtml
Abstract
The development of high performance humanoid
robots provide complex systems with which humans
must interact, and levy serious requirements on the
quality and depth of these interactions. At the same
time, developments in spoken language technology, and
in theories of social cognition and intentional cooperative
behavior provide the technical basis and theoretical
background respectively for the technical specification of
how these systems can work.
The objective of the current research is to develop a
generalized approach for human-machine interaction via
spoken language that exploits recent developments in
cognitive science - particularly notions of grammatical
constructions as form-meaning mappings in language,
and notions of shared intentions as distributed plans for
interaction and collaboration. We will demonstrate this
approach on two distinct robot platforms with human-
robot interaction at three levels. The first level is that of
commanding or directing the behavior of the system.
The second level is that of interrogating or requesting an
explanation from the system. The third and most
advanced level is that of teaching the machine a new
form of behavior. Within this context, we exploit social
interaction in two manners. First, the robot will identify
different human collaborators, and maintain a permanent
record of their interactions in order to treat novices and
experts in distinct manners. Second, the interactions are
structured around shared intentions that guide the
interactions in an ergonomic manner. We explore these
aspects of communication on two distinct robotic
platforms, the “Event Perceiver” and the Sony Aibo
ERS7, and provide in the current paper the state of
advancement of this work, and the initial lessons learned.
Introduction
Ideally, research in Human-Robot Interaction will
allow natural, ergonomic, and optimal communication
and cooperation between humans and robotic systems.
In order to make progress in this direction, we have
identified two major requirements: First, we must work
in real robotics environments in which technologists
and researchers have already developed an extensive
experience and set of needs with respect to HRI.
Second, we must develop a domain independent
language processing system that can be applied to
arbitrary domains and that has psychological validity
based on knowledge from social cognitive science. In
response to the first requirement regarding the robotic
context, we will study two distinct robotic platforms.
The first, the “Event Perceiver” is a system that can
perceive human events acted out with objects, and can
thus generate descriptions of these actions. The second
is the Sony AIBO ERS7 autonomous walking robot
running the Tekkotsu (CMU) operating system, which
provides access to a rich ensemble of sensory and motor
capabilities. From the psychologically valid language
context, we will base the interactions on a model of
language and meaning correspondence developed by
Dominey (et al. 2003) that has described both
neurological and behavioral aspects of human language,
and has been deployed in robotic contexts, and second,
on the notion of shared intentions or plans (Tomasello
2003, et al. 2006) that will be used to guide the
collaborative interaction between human and robot.
The following sections introduce the two platforms, and
the spoken language interface for command, control
and teaching the two systems.
The Event Perceiver
In a previous study, we reported on a system that
could adaptively acquire a limited grammar based on
training with human narrated video events (Dominey &
Boucher 2005, 2006). An overview of the system is
presented in Figure 1. Figure 1A illustrates the physical
setup in which the human operator performs physical
events with toy blocks in the field of view of a color
CCD camera. Figure 1B illustrates a snapshot of the
visual scene as observed by the image processing
system. Figure 2 provides a schematic characterization
of how the physical events are recognized by the image
processing system. As illustrated in Figure 1, the
human experimenter enacts and simultaneously narrates
visual scenes made up of events that occur between a
red cylinder, a green block and a blue semicircle or
“moon” on a black matte table surface. A video camera
above the surface provides a video image that is
processed by a color-based recognition and tracking
system (Smart – Panlab, Barcelona Spain) that
generates a time ordered sequence of the contacts that
occur between objects that is subsequently processed
for event analysis.
Accepted for Oral Presentation at the IEEE-RAS International Conference on Humanoid Robots
December 5-7, 2005, Tsukuba International Congress Center, Tsukuba, Japan