ISign: An architecture for humanoid assisted sign language tutoring

12Citations
Citations of this article
31Readers
Mendeley users who have this article in their library.
Get full text

Abstract

This paper investigates the role of interaction and communication kinesics in human-robot interaction. It is based on a project on Sign Language (SL) tutoring through interaction games with humanoid robots. The aim of the study is to design a computational framework, which enables to motivate the children with communication problems (i.e., ASD and hearing impairments) to understand and imitate the signs implemented by the robot using the basic upper torso gestures and sound in a turn-taking manner. This framework consists of modular computational components to endow the robot the capability of perceiving the actions of the children, carrying out a game or storytelling task and tutoring the children in any desired mode, i.e., supervised and semisupervised. Visual (colored cards), vocal (storytelling, music), touch (using tactile sensors on the robot to communicate), and motion (recognition and implementation of gestures including signs) based cues are proposed to be used for a multimodal communication between the robot, child and therapist/parent. We present an empirical and exploratory study investigating the effect of basic non-verbal gestures consisting of hand movements, body and face gestures expressed by a humanoid robot, and having comprehended the word, the child will give relevant feedback in SL or visually to the robot, according to the context of the game.

Cite

CITATION STYLE

APA

Kose, H., Akalin, N., Yorganci, R., Ertugrul, B. S., Kivrak, H., Kavak, S., … Ince, G. (2015). ISign: An architecture for humanoid assisted sign language tutoring. Springer Tracts in Advanced Robotics, 106, 157–184. https://doi.org/10.1007/978-3-319-12922-8_6

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free