Bidirectional Phrase-based Statistical Machine Translation

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1124–1132, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Bidirectional Phrase-based Statistical Machine Translation Andrew Finch NICT, Keihanna Science City, Kyoto, 619-0288, Japan andrew.finch@nict.go.jp Eiichiro Sumita NICT, Keihanna Science City, Kyoto, 619-0288, Japan eiichiro.sumita@nict.go.jp Abstract This paper investigates the effect of di- rection in phrase-based statistial machine translation decoding. We compare a typ- ical phrase-based machine translation de- coder using a left-to-right decoding strat- egy to a right-to-left decoder. We also investigate the effectiveness of a bidirec- tional decoding strategy that integrates both mono-directional approaches, with the aim of reducing the effects due to lan- guage specificity. Our experimental eval- uation was extensive, based on 272 differ- ent language pairs, and gave the surprising result that for most of the language pairs, it was better decode from right-to-left than from left-to-right. As expected the rela- tive performance of left-to-right and right- to-left strategies proved to be highly lan- guage dependent. The bidirectional ap- proach outperformed the both the left-to- right strategy and the right-to-left strategy, showing consistent improvements that ap- peared to be unrelated to the specific lan- guages used for translation. Bidirectional decoding gave rise to an improvement in performance over a left-to-right decoding strategy in terms of the BLEU score in 99% of our experiments. 1 Introduction Human language production by its very nature is an ordered process. That is to say, words are writ- ten/uttered in a sequence. The current genera- tion of phrase-based statistical machine translation (SMT) systems also generate their target word se- quences according to an order. Since the gener- ation process is symmetrical, there are two pos- sible strategies that could be used to generate the target: from beginning to end; or from end to be- ginning. Generating the target in the ‘wrong’ di- rection (the opposite direction to the way in which humans do) is counter intuitive, and possibly as a result of this, SMT systems typically generate the target word sequence in the same order as human language production. However it is not necessar- ily the case that this is most effective strategy for all language pairs. In this paper we investigate the effect of direction in phrase-based SMT decoding. For the purposes of this paper, we will refer to target word sequence generation that follows the same order as human language production as forward generation, and generation in the oppo- site direction to human language production as re- verse generation. These are often referred ”left-to- right” and ”right-to-left” respectively in the litera- ture, but we avoid this notation as many languages are naturally written from right-to-left. In earlier work (Watanabe and Sumita, 2002), it was hypothesized that the optimal direction for decoding was dependent on the characteristics of the target language. Their results show that for Japanese to English translation a reverse decod- ing strategy was the most effective, whereas for English to Japanese translation, a forward decod- ing strategy proved superior. In addition they im- plemented …

Content truncated to 500 words