Automatic evaluation of machine translation through the residual analysis

Abstract

In this study we aim at the automatic evaluation of machine translation through the residual analysis at the sentence level. We created a dataset, which covered one translation direction- a translation from an inflective language (Slovak) into an analytical language (English). BLEU (Bilingual Evaluation Understudy) as a state-of-the-art automatic metric for machine translation evaluation was used. The main contribution consists of rigorous technique (statistical method), novel to research of MT evaluation given by the residual analysis to identify differences between MT output and post-edited machine translation output.