S.S. Ge et al. (Eds.): ICSR 2010, LNAI 6414, pp. 30–39, 2010.
© Springer-Verlag Berlin Heidelberg 2010
Using a Social Robot as a Gaming Platform
F. Alonso-Martín, V. Gonzalez-Pacheco, A. Castro-González, Arnaud. A. Ramey,
Marta Yébenes, and Miguel A. Salichs
Carlos III University of Madrid, Systems Engineering and Automation Department,
281911 Leganés (Madrid), Spain
Abstract. As social robotic research advances, robots are improving their abili-
ties in Human-Robot Interaction and, therefore, becoming more human-
friendly. While robots are beginning to interact more naturally with humans,
new applications and possible uses of social robots are appearing. One of the
future applications where robots will be used is entertainment. This paper pre-
sents a social robot as a development platform for which several robotic games
have been developed. Five of these games are presented and how these games
take benefit of the robot’s HRI abilities is detailed.
Keywords: Social Robot, Edutainment, Robot Games, Robot Entertainment,
Human-Robot Interaction.
1 Introduction
Social robotics is an expanding field where research is focused on employing social
robots to perform social tasks such as entertainment, education or a combination of
both of them, edutainment.
Many robots have been designed with the aim of being used as an entertainment
robot. One of the most famous toy-robots is Aibo [1] which main goal is to behave
like a pet and play with persons for accompaniment. TOMY Company has developed
I-SOBOT. It is intended for entertainment purposes: it can be controlled by voice, it is
capable of speaking more than 200 words and phrases and it has more than 200 pre-
programmed actions. Pleo robot plays a very basic game called tug-of-war and it is
just a substitute pet. NEC is also researching on entertainment robots with Papero. It
performs dances, mimicry, riddles, quizzes, fortune telling and other games.
Other works try to mix entertainment and education in robotics leading to edutain-
ment robots. In [2], preliminary experiments on remote education have shown that
students interact with robots showing pleasure and interest. [3] describes how children
play with Roball, a plastic spherical robot, in an adaptive mode increasing and sus-
taining interaction.
Also, it has been shown that robots have a psychological effect on patients,
improving their motivation, as is demonstrated with the Paro robot. Furthermore,
children that suffer from severe disabilities use robots with the aim to learn and im-
prove their quality of life [3],[4]. Depending on the games, the robots will help the
users to develop and improve their abilities.

Using a Social Robot as a Gaming Platform 31
Almost all presented robots are equipped with a limited number of games or appli-
cations. Moreover, it is difficult to add more functionalities to them. The lack of such
software flexibility results in robots with shorter life cycles than robots which have
the capability of running more software applications. For example, in the case of toy
robots, a robot which is capable of playing several types of games can be addressed to
a wider range of people than other robot with a pool of few games.
In this work we present an easily expandable robotic platform for the development
of board games as well as educational applications. The platform consists of the robot
Maggie [5] and its software architecture as the base for the creation of several game
skills that allow the robot to play board games.
The paper continues in the next section with a short description of the robot
Maggie. We describe the hardware and software platforms used and we detail which
are the interaction capabilities of the robot. Section 3 presents the interaction mecha-
nisms of the robot. In section 4 we discuss the robot as gaming platform and present a
description of five of the games that we have developed. Finally, in section 5 a brief
conclusion and future issues are discussed.
2 Description of the Robot
The robot Maggie [5] is a platform for studying human-robot interaction (HRI). The
development of the robot is focused in finding new ways to adapt the robotics poten-
tial to provide to human users new ways of working, learning and entertaining.
2.1 Hardware
Maggie is designed as a 1.35 meters tall girl-like doll. Its base is motorized by two
actuated wheels and a caster wheel. The base is equipped with 12 bumpers, 12 infra-
red optical sensors and 12 ultrasound sensors. Above the base, a laser range finder
(Sick LMS 200) has been added. The upper part of the robot incorporates the interac-
tion modules. On top of the platform, there is a robot head with an attractive design.
The head has two degrees of freedom, while each arm has one degree of freedom
(DoF). These features are illustrated in Fig 1.
Maggie is controlled by a main computer hidden inside her body. In the computer
resides the software architecture of the robot. For image acquisition, the robot has a
camera located in the robot’s mouth. The camera is a Logitech QuickCam Pro 9000.
The robot has touch sensors on the surface of the body and a touch screen situated on
the chest. Finally, inside the head, an RFID antenna is placed to identify objects.
2.2 Software Architecture
The software architecture of the robot is the Automatic-Deliberative (AD) architec-
ture [6]. AD is composed of two levels, the automatic level and the deliberative level.
The automatic level is where the low-level control is done: in the automatic level,
the modules that provide communication and control of the sensors, motors and
other hardware are located. At the deliberative level, reasoning and decision processes
are placed.

32 F. Alonso-Martín et al.

Fig. 1. The hardware equipping Maggie
The essential component of the AD architecture is the skill. A skill is an entity with
the capacity of reasoning, processing data or carrying out actions and with the capa-
bility of communicating with other skills. For example, the laserSkill manages the
laser range finder of the robot, formats its data and shares it with the rest of the skills.
The other skills can benefit from the data obtained from the laserSkill to, for example,
build a map and share it with other skills. A detailed description of the AD architec-
ture can be found in [6] and [7].
3 Interaction with Maggie
Maggie is a robot designed and created to interact with humans, therefore its goal is to
explore the mechanisms of interaction with humans. At present, the robot has several
mechanisms of interaction that we analyse here. All of the following interaction
mechanisms can be used by the games to improve the perceived experience of the
human while it is playing with the robot.
a) Voice System: The most important interaction mechanism of the robot is the
voice system. This system allows the robot to speak and to listen to humans. The voice
system is composed of a set of skills that gives the robot a complex and powerful sys-
tem of communication. The main voice skills are: Automatic Speech Recognition

Using a Social Robot as a Gaming Platform 33
(ASR), Emotional Text To Speech (eTTS), Speaker Identification, Voice Tracker and
DialogueSkill based in VoiceXML. The voice generator and voice recognition are
provided by a professional software vendor Loquendo wrapped into the global archi-
tecture of Maggie as a skill.
The robot voice is generated by a Text To Speech (TTS) system which allows the
robot to convert any written text to human voice. It can generate voices in Spanish
and English with high quality and several emotions. The voice is clear and easily
understandable by humans. Moreover varying the voice tone, the robot can communi-
cate with expressions and emotions as happiness, sadness, tranquillity or excitement.
It is also possible to generate laughter, yawning and sighing, which are very useful
in games.
The robot is also able to understand what we say. The human communicates with
the robot through a wireless microphone and it is understood by Maggie with a speech
recognizer using a grammatical based knowledge system. Using this skill Maggie can
understand Spanish, American English and British English. Any human can talk to
the robot in natural language and a training phase it is not needed.
Currently, all the interactions are performed following two paradigms of HRI, the
master-slave and the peer-to-peer (P2P) paradigms. Depending on the context,
the interaction will lead to one or the other paradigm. In the master-slave paradigm,
the human acts as the master and the robot obeys the commands expressed by the
human. In the P2P both human and robot interact as equals. The first is used in nor-
mal contexts where Maggie acts as a personal assistant. The second is mainly used in
games contexts where the human and the robot play games acting as rivals or partners
depending on the game.
Maggie is also able to identify the person which is talking to her using a previous
recorded voice-print database. This capability is called Speaker Verification and is
powered by the Loquendo speech system.
All these skills are controlled by the dialogueSkill (in a higher level of abstraction)
and it is based in VoiceXML.
b) Touch sensors: Maggie can sense when a person touches certain parts of its
body (head, arms and upper body). The robot has a dozen capacitive touch sensors
placed over her body. This skill is always running and always detects when a human
touches a sensor. It is very useful in games as another interaction mode.
c) Computer Vision: The robot has a camera in its mouth which enables her to
“see” its environment. The vision system captures the images from the camera and
processes them with the OpenCV library. The processed data can then be used by
several skills, for example: detecting a person, counting the number of people in the
environment, detecting the game board, etc.
d) Radio Frequency Identification (RFID): Another mechanism to interact with
the environment is using Radio Frequency Identification (RFID) tags [8]. Altough it is
not a naturally type of interaction between humans, it allows the robot to identify and
retrieve information about several objects that are presented to the robot by the hu-
man. Maggie has a RFID reader in her body. This reader can read radio frequency
tags inserted in objects. When the human presents the object to the robot, the robot is
not only able to identify the object itself, but also is able to retrieve more information
related to it. We have developed several skills that use this kind of interaction: reading
and retrieving information about certain products like drugs and toys. Note that many

34 F. Alonso-Martín et al.
utilities developed with this identification system may be processed either by the
vision system or the RFID detection system. While the first is similar to the human
sensors, with the second it is easier to develop new applications which are more
robust to certain conditions like low light rooms, etc.
4 Gaming Platform
Maggie is able to play several games thanks to the design of her software architecture.
Since AD enables the construction of new skills that make use of previously built
skills, the development of new games becomes a task where most of the work has
already been done by other skills. For example, to create a new game, it is possible to
use the vision skills data to locate the game board and process the current state of the
game. After that, the new game skill can calculate the next movement and use the
communication skills to tell the human which is going to be the robot’s next move-
ment. In other words, creating a new game consists in developing the algorithm of the
new game, and after that complement it with the necessary skills to allow the robot to
interact with the world and the human player.
The robot interaction capabilities can also be used to allow the robot to play board
games with humans in the same manner humans play with other humans. For exam-
ple, the voice system is not only used during the games. During the complete opera-
tion of the robot it can be operated by voice. The robot understands several dialogues
that allow the user to activate and deactivate by voice the available skills like the
games or other robot utilities. Our aim to have the robot completely operated by voice
pursues producing the feeling of interacting with other person.
The following sections show some of the games we have developed until so far to
test the interaction capabilities of the robot and are a an example of how to create a
complex gaming robotic platform by the addition of several aggregated skills. Several
of the described games are based in classical board games.
4.1 Peekaboo
This game is the robotic version of the classical Peekaboo game. We first developed
this game to test a face detection skill and then was integrated into the pool of games
that Maggie can play. The human Peekaboo version is a game played with babies in
which the adult hides her face with her hands or with an object. In the robotic version,
a person hides her face from the robot in the same way as the classical version.
The purpose of the game is hiding the face to make it undetectable by the robot's
computer vision system. If the robot does not detect any faces, then says that she can
not see anybody. When the person shows her face to the robot, then tells that now she
is seeing her. More than one person can play the game at the same time. In this case,
the robot tells to the group the number of faces she is seeing.
4.2 Guessing a Character
In this game the human player must think on a fictional or real character. The
robot asks several questions to the human until it is able to guess the character. The

Using a Social Robot as a Gaming Platform 35
questions require only yes/no answer. Usually in less than 20 questions the robot
guesses the character.
Before starting the game, the robot describes the rules of the game and how the
player must interact with the robot. Once the robot has finished the description of the
game the player must have decided what the character will be. The game starts when
the robot starts asking questions about the character. The robot asks the human using
its voice system. The human also responds by voice and the robot has to analyse and
detect the responses of the human.
The game is implemented as a skill that uses the robot’s built-in wifi connection to
connect it to a public web server (Akinator1). This server has a database of many
characters and the intelligence to relate the answers of the human with all the possible
characters.
Using machine learning techniques and artificial intelligence the web service se-
lects questions so that the answer eliminates as many potential characters as possible.
The implementation of the algorithm is based on a search tree. Each question
eliminates the maximum number of options, filtering the branch that has more
children
4.3 Tic-tac-toe
In this game, the robot and the human play the game tic-tac-toe. The game is played
on a board of 3x3 cells. The robot can play either with crosses or circles. The player
can start the game or let the robot do it. If the human starts the game, putting down a
counter of one kind, the robot will use the other kind of counters. If the robot starts, it
always chooses crosses.
The game is performed in the following way. Supposing the robot starts the game,
it chooses a position and it tells it to the human. Because the robot lacks hands,
the human must put the counter for it in the position the robot has asked to. After that,
the human turn begins. The human chooses a free position, puts the counter on it and
lets the robot know that he has finished. Once the human turn has finished, the robot
analyzes the board with its vision system. After the analysis of the board, the robot
has an updated status of the game so it can perform the next move.
The game ends when one of the players has managed to put three consecutive
counters on the board (i.e. it has formed a row). In this case that player has won the
game. The game also finishes if the board ends full and neither the robot nor the hu-
man player have managed to put three consecutive counters. In this case the game
finishes in a draw.
The robot uses its vision system to recognize the game board. For that reason it
must be mounted on a table near enough to let the robot see it (usually at 1m height
and about 20cm from the robot).
The interaction between the robot and the human is done by voice. The robot tells
the human the rules of the game, asks the human to put a counter in a certain position
on the game board and updates the human with the state of the game (I.e. is finished,
who has won, etc.). The human must warn the robot when he has finished his turn.


1
Akinator is the name of the game, http://en.akinator.com/

36 F. Alonso-Martín et al.


Fig. 2. Maggie playing to tic-tac-toe Fig. 3. Recognizing the board game
In order to recognize the play area (Fig. 3), the robot uses a vision machine algo-
rithm that threshold the image and finds a black square which is the frame of the
game board (Fig 4a). Once it has found the game board, it corrects the image from a
perspective view to a plant view (Fig 4b). In the plant view the algorithm recognizes
the game counters placed in the game board (Fig. 4c). These algorithms are based on
the OpenCV libraries.


(a) (b) (c)
Fig. 4. (a) Board game (color perspective image); (b) Board game (threshold perspective
image); (c) Image corrected in plant view
Once the position of the game counters is defined, it is necessary to apply an
appropriate algorithm to decide the next move. The game algorithm is based on mini-
mizing the “damage” that you can receive from the adversary and maximize your
chances of winning.
4.4 Hangman
In this game, the human thinks of a word and the robot tries to guess it asking ques-
tions and processing a few clues given by the human. The human writes, on a piece of
paper, as many underscores as letters the word has. After that, the human puts the
piece of paper on the table to allow the robot to view and count the number of under-
scores of the word. The human can give the robot more clues, for example one or
more letters of the word, written in the exact position of the word. For example, in the
word “robot”, the human can write a “b“, but it must be placed just above the third
underscore, which corresponds with the letter “b“ of “robot“. Counting the number of
underscores, the robot is able to know the number of letters of the word. If the human
gives her a clue in the form of a letter, the robot has then more information of the
word and can reduce the list of possible words.

Using a Social Robot as a Gaming Platform 37
When the human puts the paper with the underscores on the table, he has to tell the
robot to start the game. Then, the robot tries to guess the complete word. To do this
the robot has several rounds. In each round the robot must guess a letter of the word.
If the proposed letter is in the word, the human must write it on the game board, just
above the corresponding position of the word. The game ends when the robot guesses
the word or when the robot reaches a maximum number of failures.
Like in the tic-tac-toe game, it is necessary to put the table near the robot to allow
it to see the game board (Fig. 5). The human player is responsible for thinking of the
word, and writing the underscores corresponding to the number of letters of the word
in the game board. The robot is responsible of guessing the word. In each round the
robot asks for a letter, and the human player writes this letter above the underscores
that correspond to the letter in the word (in the case this letter is in the word). For
example, in the word “robot”, if the robot asks for the letter “o”. The human must
write an “o” above the second and the fourth underscores. Doing that, the robot is
able to detect that she has guessed the letter and to detect that the letter appears two
times in the word. If the letter is not part of the word a failure is considered. The
maximum number of failures is six. The game ends when the robot guesses the word
or when the six failures have been reached.
In each turn, the user must write clearly and with
black marker the letters that the robot has guessed. Once
the human has done this, he warns the robot to make
another attempt using a voice command. After that, the
robot analyzes the game board, counting the number of
underscores and the letters that it has guessed. Again,
both for character recognition (OCR) and to find the
board game we use computer vision techniques based on
the OpenCV libraries.
In Fig.6a, we can see the game board in the top right
corner. In Fig. 6b the robot analyzes the image and de-
tects the game board (square with big black border). In
the Fig. 6c the robot has obtained the rectified image
from the playing area (plant view) and has detected and
identified the letters written on it.


Fig. 6. (a) Table with the board game, (b) Maggie detecting the board game; (c) OCR in the
rectified image

Fig. 5. Maggie playing
hangman
(a) (b) (c)

38 F. Alonso-Martín et al.
The algorithm used in the game is based on finding the words that can match with
the current state of the game, those words are in a dictionary with the most common
words of the language. We have two dictionaries of words, one in English and another
in Spanish, each one with approximately 100,000 words. These words are the most
common in both languages.
The human part of human-robot interaction in this game consists in writing in the
board game and talking to indicate the end of each turn. In the other part the robot
interacts with the human reading the writing of the human with its vision system and
making questions to the human related to the game.
4.5 Animal Quiz
The aim of this game is to study the interaction between the robot and children using
the voice system and RFID sensors.
In order to play to animal quiz we have ten soft toy animals with RFID tags inside
of them. Each soft toy is an animal of one color and has a single unique name. There
are not two soft toys that are the same animal, have the same color or have the same
name. The game consists in Maggie asking a child to bring her one of the soft toys.
To do that the robot asks for one of the properties of the soft toy (animal, color or
name). The child picks the corresponding soft toy and brings it to the robot. The child
gives the soft toy to the robot by bringing it near to the nose of the robot, where the
RFID reader is located. This allows the robot to detect if the child has brought the
correct soft toy or not. Writing and reading RFID tags in Maggie is explained in [8].
If the child has not understood the question she can ask the robot to repeat it.
When the game has finished, the robot tells the number of right and wrong answers.
Only the first time the child shows the toy in each question is counted as hit or failure.
When the robot asks for a soft toy, it waits until an RFID tag is detected. The toy
must be placed close to the robot nose, typically 20 cm. The robot compares the num-
ber stored in the tag with the right answer in order to know if the toy is the correct one
or not. The child may try to guess the soft toy again and again until he gets right.
5 Conclusions
In this paper we have presented a social robot with many interaction capabilities and
its use as a gaming platform. We have presented the robot hardware and software
architecture and its interaction skills. The presented robot software architecture
facilitates the creation of robot applications such as games by composing previously
developed skills into new ones. It is, for example, easy to create a game that uses the
presented interaction capabilities of the robot to interact with the environment and
with the human player.
A social robot with the capability of running applications is able to adapt to scenar-
ios which were not initially intended or designed for it. In this way the life cycle of
the robot can be increased considerably thanks to the constant development of new
applications that at the same time could enlighten new scenarios and areas to where
the robot could be used. In this case, the new area is the gaming area.

Using a Social Robot as a Gaming Platform 39
Following the exploration of this area, our future work consists on conducting
experiments to study how people react and behave when they are playing with a robot
that tries to behave in the same way a human would do in such game situations. The
first data seems to indicate that people tends to be more involved when a robotic char-
acter shows emotions during the game. Also, our preliminary results show that robots
with more interaction capabilities make people feel more comfortable and, as a result,
they tend to spend more time playing with the robot.
References
1. Fujita, M.: On activating human communications with pet-type robot AIBO. Proceedings of
the IEEE (2004)
2. Yorita, A., Hashimoto, T., Kobayashi, H., Kubota, N.: Remote Education Based on Robot
Edutainment. Communications in Computer and Information Science 44(Part 3), 204–213
(2009), doi:10.1007/978-3-642-03986-7_24
3. Robin, B., Dautenhahn, K.: Interacting with robots: Can we encourage social interaction
skills in children with autism? Accessibility and Computing (2004) ISSN:1558-2337
4. Cook, A.M., Meng, M.Q.-H., Gu, J.J., Howery, K.: Development of a robotic device for
facilitating learning by children who have severe disabilities. Neural Systems and Rehabili-
tation Engineering (2002) ISSN: 1534-4320
5. Salichs, M.A., Barber, R., Malfaz, M., Gorostiza, J.F., Pacheco, R., Rivas, R., Corrales, A.,
Delgado, E., García, D.: Maggie: A Robotic Platform for Human-Robot Social Interaction.
Robotics, Automation and Mechatronics (2006) ISBN: 1-4244-0024-4
6. Barber, R., Salichs, M.A.: A new human based architecture for intelligent autonomous
robots. In: IFAC Symposium on Intelligent Autonomous Vehicles, Sapporo, Japan (Septem-
ber 2001) ISBN:0-08-043899-7
7. Rivas, R., Corrales, A., Barber, R., MA: Robot skill abstraction for AD architecture. In: 6th
IFAC Symposium on Intelligent Autonomous Vehicles (2007)
8. Corrales, A., Rivas, R., Salichs, M.A.: Integration of a RFID System in a social robot.
Communications in Computer and Information Science 44(Part 2), 63–73 (2009),
doi:10.1007/978-3-642-03986-7_8