Implementing statistical machine translation into mobile augmented reality systems

Abstract

A statistical machine translation (SMT) capability would be very useful in augmented reality (AR) systems. For example, translating and displaying text in a smart phone camera image would be useful to a traveler needing to read signs and restaurant menus, or reading medical documents when a medical problem arises when visiting a foreign country. Such system would also be useful for foreign students to translate lectures in real time on their mobile devices. However, SMT quality has been neglected in AR systems research, which has focused on other aspects, such as image processing, optical character recognition (OCR), distributed architectures, and user interaction. In addition, general-purpose translation services, such as Google Translate, used in some AR systems are not well-tuned to produce high-quality translations in specific domains and are Internet connection dependent. This research devised SMT methods and evaluated their performance for potential use in AR systems. We give particular attention to domain-adapted SMT systems, in which an SMT capability is tuned to a particular domain of text to increase translation quality. We focus on translation between the Polish and English languages, which presents a number of challenges due to fundamental linguistic differences. However, the SMT systems used are readily extensible to other language pairs. SMT techniques are applied to two domains in translation experiments: European Medicines Agency (EMEA) medical leaflets and the Technology, Entertainment, Design (TED) lectures. In addition, field experiments are conducted on random samples of Polish text found in city signs, posters, restaurant menus, lectures on biology and computer science, and medical leaflets. Texts from these domains are translated by a number of SMT system variants, and the systems’ performance is evaluated by standard translation performance metrics and compared. The results appear very promising and encourage future applications of SMT to AR systems.