Sinuhe – Statistical Machine Translation using a Globally Trained Conditional Exponential Family Translation Model

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1027–1036, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Sinuhe — Statistical Machine Translation using a Globally Trained Conditional Exponential Family Translation Model Matti K¨a¨ari¨ainen Department of Computer Science FI-00014 University of Helsinki, Finland matti.kaariainen@cs.helsinki.fi Abstract We present a new phrase-based con- ditional exponential family translation model for statistical machine translation. The model operates on a feature repre- sentation in which sentence level transla- tions are represented by enumerating all the known phrase level translations that occur inside them. This makes the model a good match with the commonly used phrase extraction heuristics. The model’s predictions are properly normalized prob- abilities. In addition, the model automati- cally takes into account information pro- vided by phrase overlaps, and does not suffer from reference translation reacha- bility problems. We have implemented an open source translation system Sinuhe based on the proposed translation model. Our experi- ments on Europarl and GigaFrEn corpora demonstrate that finding the unique MAP parameters for the model on large scale data is feasible with simple stochastic gra- dient methods. Sinuhe is fast and mem- ory efficient, and the BLEU scores ob- tained by it are only slightly inferior to those of Moses. 1 Introduction In current phrase-based statistical machine transla- tion systems such as Moses1 (Koehn et al., 2007), the translation model is defined in terms of phrase pairs (biphrases) extracted from a bilingual cor- pus as follows. The corpus is first word-aligned using a word alignment heuristic (Och and Ney, 1Throughout this paper, we refer to Moses for concrete- ness, but most of the discussion applies to other standard phrase-based statistical machine translation systems as well. 2003). The phrase extraction heuristic then ex- tracts all the biphrases that are compatible with the word alignment (Och et al., 1999). This way, each sentence pair may generate any number of potentially overlapping biphrases. However, when defining the phrase-based sentence level transla- tion model, phrase overlaps are explicitly disal- lowed: The source sentence is segmented into dis- joint phrases, which are translated independently using conditional phrase-level translation models that have been estimated from extracted biphrase counts. The disparity between the phrase extraction heuristic and the use of the extracted biphrases can be addressed in at least three ways. One approach is to simply ignore the disparity as is done, e.g., in Moses. While empirically succes- ful, this approach is hard to justify theoretically, and begs the question of whether more principled methods might lead to better translation results. The other extensively studied approach is to re- place the phrase extraction heuristic with a method that better matches the use of the extracted phrases (see, e.g., (Marcu and Wong, 2002; DeNero et al., 2008) and the references therein). While theo- retically sound, this approach is computationally challenging both in practice (DeNero et al., 2008) and in theory (DeNero and Klein, 2008), may suf- fer from reference reachability problems (DeNero et al., 2006), and in the end may lead …

Content truncated to 500 words