Teaching Sequences to a Social Robot by
Voice Interaction
Javi F. Gorostiza and Miguel A. Salichs
Abstract— In this paper a sequence manager system for
Robot Teaching is presented. This system allows the user to
edit, execute and debug the sequence by means of speech with
a multimodal social robot. The main goal of the paper is to
make human-robot interaction easier for the non-expert users.
To achieve this we are designing a game for children where they
play to teach to the robot a sequence of actions and conditions
by means of human-robot interaction. A Sequence Function
Chart (SFC) representation for sequence implementation is
proposed. This representation is transparent for the non-expert
user, that just uses natural language to interact with the robot.
We think that this work will be notable for the development of
social robots in our society and to close these robots to common
people.
I. INTRODUCTION
Social robots are going to take part of the common life of
humans in the developed societies. In 2006 around 3,540,000
service robots were made where only 950,000 were expected
[5] Social robots form a new class of computer-based enter-
tainment that is beginning to become commercially practical.
We focus on how robots are going to learn new tasks
from human, and how to make robots that can educate and
entertain the user. The Robot Teaching issue merges concepts
of robot learning and human - robot interaction, where the
user teaches something to the robot. The major directions in
robot learning have been using human learning models for
non-supervised learning or programming by demonstration.
These modus operandi have given very good results: the
control architecture proposed in [2] allows the robot to infer
the correct goal in a human-robot collaboration scenario;
in [9] a system for robot skill acquisition from kinesthetic
demonstrations is presented. The implemented system treats
low-level motion control of the DOF of a humanoid robot.
In other works about programming by demonstration it
is proposed a solution of how to imitate (the so called
correspondence problem) and what to imitate.
Here we propose a different paradigm where the user teaches
or programs complex sequences of actions and conditions in
a social robot by speech. A particular goal to achieve is to
make a game for children where the child can play to teach
sequences to the robot. This system will allow to teach new
and complex tasks to the robot in an easy and natural way
for non-experts users.
We have chosen a Sequence Function Chart standard [6]
Javi F. Gorostiza is with RoboticsLab, Escuela Polite´cnica Superior,
Universidad Carlos III de Madrid, av. Universidad, 30, Leganes, Spain
jgorosti@ing.uc3m.es
Javi F. Gorostiza is with RoboticsLab, Escuela Polite´cnica Superior,
Universidad Carlos III de Madrid, av. Universidad, 30, Leganes, Spain
msalichs@ing.uc3m.es
to represent the sequence of actions and conditions in the
robot. The sequence can be created, executed and debugged
at runtime. Other works have proposed similar representation
frameworks. In [11] a high level multi-robot programming
representation based on the semantics of Petri nets is imple-
mented. But in this work the expert developer has to program
the plans by hand. The plans managed involve a small set of
actions and conditions that are very constrained to a model
of the environment.
In section II it is made an introduction to SFC standard.
In sections III, IV and V the personal robot Maggie and its
control architecture are presented as the researching platform
of the present paper. In the next section VI it is presented
the skill that allows the user to edit a sequence by voice
interaction. We present the first experimental results in some
cases where the user edits the sequence by means of speech
interaction in section VII.
II. SEQUENCE FUNCTION CHART STANDARD
AND SEQUENCES
SFC is used here for sequence represention. This graphical
programming language is defined by IEC 61131-3 standard
for Programmable Logic Controllers (PLC’s). Based in the
so called GRAFCET (Graphe de commande etape-transition)
it is an evolution of a Petri Net that allows the representation
of a Sequential System, that is, a system where the outputs
do not depend only on the inputs, but on the internal state
of the system, too.
A sequence can be defined as a net where each node is an
action or a condition. An action is defined as any process
or effect of the robot activity. A condition is defined as a
functional entity that can take one of two values: true or
false, depending on certain variables of the robot or the
environment, where we include the user. Then, a sequence is
a set of actions, a set of conditions and a set of links between
these actions and conditions. SFC is based on two main
entities: steps and transitions. The former are associated with
actions and the latter with conditions. A SFC is a bipartite
graph, so the connections between nodes of the same type
are not allowed, that is, one step cannot be directly linked
to another step and idem with transitions.
A. Steps and Actions
Symbolically steps are represented as squares with an
alphanumerical label. Next to the square the associated
actions are defined with a label text. Here we consider that
just one action is associated with the step. At a given instant
a step may be either active or inactive, what it’s represented