A serious game for learning portuguese sign language - “iLearnPSL”

Abstract

Several thousands of people in Portugal use Portuguese sign language (PSL). Children belonging to this community have difficulties with communication and learning processes. There are some applications that focuses on teaching/learning PSL but with a limited interaction with the user, such as presentation of images of a hand gesture picture or avatar where the user must mimic the respective PSL hand gesture. Some other applications are more advanced using sensors to detect movements and gestures performed by the user. However, these applications do not implement an automatic interaction with the user of PSL. Following this trend, the main idea of this project is to develop a solution where deaf child can learn PSL in an interactive, dynamic and funny way. Therefore, this paper describes the first insights in the development of an interactive and didactic virtual game tool to learn the Portuguese Sign Language, in particular the numbers from 0 to 9. The alphabet and colors will be also included in the future. The target group considers deaf children in the first cycle of education level (primary Portuguese school). The developed game promotes the automatic interaction with the user through the Leap Motion controller. The system captures the hands and fingers gestures performed by the user and the graphical interface returns the adequate feedback. The preliminary results obtained show a good level of user experience with therapists.