Tracking improvement from statistical to neural machine translation: An er-ror-based evaluation of google translate

Abstract

This study investigated the quality of Google Translate outputs after the system shifted from statistical machine translation (SMT) to neural machine translation (NMT). The examples originally analyzed by Al-Zebary (2012), when Google Translate operated under the SMT framework, were retranslated through Google Translate in 2023 and 2025. The resulting outputs were examined for both lexical and structural issues, and a comparative analysis was conducted across the three sets of translations to determine whether the problems previously identified continue to persist. The findings revealed that Google Translate has undergone substantial improvement under NMT, where many of the errors reported in earlier SMT output such as deletion, addition, misinterpretation of homographs, and literal translation of idioms have been significantly reduced. There was also a noticeable improvement from the 2023 NMT outputs to those produced in 2025, indicating continued refinement in Google Translate’s handling of English–Arabic translation. However, the study shows that unnatural phrasing continues to occur in linguistically complex contexts, especially those involving cross-linguistic structural mismatches. The results confirm that despite notable progress, machine translation still requires human intervention to ensure accuracy, naturalness, and contextual appropriateness in English–Arabic translation.