Cultural references and machine translation: a methodology for evaluation. The case of administrative texts in the area of migration

Abstract

Cultural references constitute one of the great challenges for machine translation (MT). Despite the existence of numerous ad hoc studies on this technology, the number of cases where it can be applied exceeds the limits of current research. MT of a language's own cultural manifestations is one of the fields that have been little explored so far. Consequently, there is a need to review the evaluation methods often used to determine the validity of the texts produced by MT, with a focus on cultural referents on the field of migration. In this line, the article presents a methodological proposal for MT evaluation which is based primarily on the following qualitative data: fluency, accuracy, and acceptability. To these, quantitative data is added on the perception of these same criteria. First, the cultural references are contextualised in the framework of MT and artificial intelligence. Then, the different evaluation methods of automatic and manual evaluation are presented, with the definition of a specific methodology for the evaluation of cultural references. To illustrate this methodology from a practical point of view, a case of evaluation of cultural references is shown by means of an exploratory study carried out with administrative texts in the field of migration.