Recurrent Neural network based rule sequence model for statistical machine translation

Abstract

The inability to model long-distance depen-dency has been handicapping SMT for years. Specifically, the context independence as-sumption makes it hard to capture the depen-dency between translation rules. In this paper, we introduce a novel recurrent neural network based rule sequence model to incorporate arbi-trary long contextual information during esti-mating probabilities of rule sequences. More-over, our model frees the translation model from keeping huge and redundant grammars, resulting in more efficient training and de-coding. Experimental results show that our method achieves a 0.9 point BLEU gain over the baseline, and a significant reduction in rule table size for both phrase-based and hierarchi-cal phrase-based systems.