On evaluating the effects of feedback for sign language learning using explainable AI

Abstract

Computer Aided Sign Language Learning (CASLL) is a recent and promising field of research which is made feasible by advances in Computer Vision and Sign Language Recognition. The importance of feedback for language learning has been established by many research works. In this work, we introduce SignGuru a chat-bot based AI tutor that can provide fine-grained feedback to learners of American Sign Language. SignGuru provides feedback directly relating to the fundamental concepts of ASL using a modular and explainable AI. The chatbot is designed not only as an interactive for learners to choose a curriculum and go through the interactive learning process, but also to perform retention and execution tests. The usability and utility of SignGuru has been validated by a userstudy using 14 ASL signs with 26 users. We demonstrate the fully functioning application with a variety of different curriculum.