Deep Learning-based Arabic Sign Recognition System for Automated Communication with Hearing Impaired Individuals

Abstract

Arabic Sign Language (ArSL) is used by individuals who are hard of hearing or deaf in Arab countries, as well as others around the world who use it for religious purposes. for the need for automated systems to facilitate the learning and communication of ArSL is therefore significant. Such systems would allow people to learn Arabic Sign Language and use it to communicate among themselves and with the surrounding community. This paper presents the development of an automatic recognition system capable of accurately identifying Arabic signs through hand gestures. In this paper, two Residual Network (ResNet) Configurations, Version 1 (V1) and Version 2 (V2), are proposed and detailed. The proposed ResNet V1 achieved an average accuracy of 98.83%, while ResNet V2 achieved an average accuracy of 98.84%. The results described in this paper far exceed those reported in the extant literature. The high accuracy of the proposed system shows the potential for integrating the system with education tools and assistive technologies for people with special needs.