Length-constrained Neural Machine Translation using Length Prediction and Perturbation into Length-aware Positional Encoding

  • Oka Y
  • Sudoh K
  • Nakamura S
N/ACitations
Citations of this article
7Readers
Mendeley users who have this article in their library.

Abstract

Neural machine translation often suffers from an under-translation problem owing to its limited modeling of the output sequence lengths. In this study, we propose a novel approach to training a Transformer model using length constraints based on length-aware positional encoding (PE). Because length constraints with exact target sentence lengths degrade the translation performance, we add a random perturbation with a uniform distribution within a certain range to the length constraints in the PE during the training. In the inference step, we predicted the output lengths from the input sequences using a length prediction model based on a large-scale pre-trained language model. In Japanese-to-English and English-to-Japanese translation, experimental results show that the proposed perturbation injection improves the robustness of the length prediction errors, particularly within a certain range.

Cite

CITATION STYLE

APA

Oka, Y., Sudoh, K., & Nakamura, S. (2021). Length-constrained Neural Machine Translation using Length Prediction and Perturbation into Length-aware Positional Encoding. Journal of Natural Language Processing, 28(3), 778–801. https://doi.org/10.5715/jnlp.28.778

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free