Prototyping and preliminary evaluation of sign language translation system in the railway domain

Abstract

This paper presents the prototype and the preliminary evaluation of an automatic translation system developed in the LIS4ALL project. The system domain is the corpus of railway station announcements in Italian. The output of the system is a 3D animated avatar that signs announcements in Italian Sign Language. The preliminary evaluation, which measures the accuracy of the translations at the sentence level, relies through the BLEU-RAC4 metric, a variant of the traditional BLEU metric used to evaluate Machine Translation, specifically designed for sentence level evaluation. The aim of the evaluation is to compare the LIS4ALL translation outputs with the human counterparts.