TUD-BISINDO: A new dataset and its recognition system using YOLO

Abstract

This study addresses the urgent need for digital inclusivity by developing a high-precision, real-time recognition system for Bahasa Isyarat Indonesia (BISINDO). The main new idea in this study is the creation of the Telkom University Database (TUD)-BISINDO, a robust, diverse collection of data designed to address the limitations of current sign language databases, such as insufficient variation in environments and camera angles. The TUD-BISINDO was created using 1,040 original images and added 780 more to address issues such as lighting, angles, and hand features that were often found in earlier datasets. The YOLOv8l model, improved with the AdamW optimizer and a flexible learning rate, performed exceptionally well with a mAP50 of 99.30%, mAP50-95 of 85.40%, 99.80% precision, and 99.70% recall. These results demonstrate that the model significantly outperforms the previous YOLOv5 baseline across all primary metrics. The model has outstanding precision in recognizing real-time finger movements. However, complex gestures, including the G and Z letters, require further refinement. This research enhances sign language recognition technology, encouraging inclusion and improving accessibility for real-time communication. Future studies should focus on diversifying the dataset and maximizing performance in challenging conditions.