Using YOLOv5 Algorithm to Detect and Recognize American Sign Language

Abstract

Sign language is a system of communication using visual gestures and signs, and is generally used by people who have speech or hearing difficulties. To make those people included in the verbal communication community it's important to under-stand the gesture they're using to communicate. Often people who do not use the gesture in real life could not understand what the gesture represents. In this paper, we're proposing a solution to detect the alphabet and numbers that are each gesture providing. There are already some methods proposed that use deep learning for recognizing sign language. However, the effective uses of these models are limited. We are proposing a YOLOv5 based solution as it's lightweight, fast, and has good accuracy. We use a benchmark data set (MU_HandImages_ASL) to train and evaluate the model. We achieved 95% precision, 97% recall, 98% map@0.5, and 98% map@0.5:0.95 score which is adequate to recognize the gesture in real-time.