Deep learning and collaborative training for reducing communication barriers with deaf people

Abstract

Building bridges to facilitate communication between deaf and hearing people is a pending issue that can no longer be postponed in a technologically advanced society. In this work, we propose a multiplatform service that allows gradually building a collaborative platform to bring the interests of deaf and hearing people closer together. The proposal consists of a client-server architecture with two main parts: a multiplatform interface for video capture and playback, and a sign language recognizer system composed of a skeleton data extractor and a Graph Convolutional Neural Network. The result is a proof-of-concept of the first thematic dictionary accessible through sign language.