Language Models and Reranking for Machine Translation

Proceedings of the Workshop on Statistical Machine Translation, pages 150–153, New York City, June 2006. c⃝2006 Association for Computational Linguistics Language Models and Reranking for Machine Translation Marian Olteanu, Pasin Suriyentrakorn and Dan Moldovan Language Computer Corp. Richardson, TX 75080 {marian,psuri,moldovan}@languagecomputer.com Abstract Complex Language Models cannot be eas- ily integrated in the first pass decoding of a Statistical Machine Translation system – the decoder queries the LM a very large number of times; the search process in the decoding builds the hypotheses incremen- tally and cannot make use of LMs that analyze the whole sentence. We present in this paper the Language Computer’s system for WMT06 that employs LM- powered reranking on hypotheses gener- ated by phrase-based SMT systems 1 Introduction Statistical machine translation (SMT) systems com- bine a number of translation models with one or more language models. Adding complex language models in the incremental process of decoding is a very challenging task. Some language models can only score sentences as a whole. Also, SMT de- coders generate during the search process a very large number of partial hypotheses and query the language model/models 1. The solution to these problems is either to use multiple iterations for decoding, to make use of the complex LMs only for complete hypotheses in the search space or to generate n-best lists and to rescore the hypotheses using also the additional LMs. For 1During the translation of the first 10 sentences of the de- vtest2006.de dataset using Phramer and the configuration de- scribed in Section 3, the 3-gram LM was queried 27 million times (3 million distinct queries). the WMT 2006 shared task we opted for the rerank- ing solution. This paper describes our solution and results. 2 System Description We developed for the WMT 2006 shared task a sys- tem that is trained on a (a) word-aligned bilingual corpus, (b) a large monolingual (English) corpus and (c) an English treebank and it is capable of translat- ing from a source language (German, Spanish and French) into English. Our system embeds Phramer2 (used for mini- mum error rate training, decoding, decoding tools), Pharaoh (Koehn, 2004) (decoding), Carmel 3 (helper for Pharaoh in n-best generation), Char- niak’s parser (Charniak, 2001) (language model) and SRILM4 (n-gram LM construction). 2.1 Translation table construction We developed a component that builds a translation table from a word-aligned parallel corpus. The com- ponent generates the translation table according to the process described in the Pharaoh training man- ual5. It generates a vector of 5 numeric values for each phrase pair: • phrase translation probability: φ( ¯f|¯e) = count( ¯f, ¯e) count(¯e) , φ(¯e| ¯f) = count( ¯f, ¯e) count( ¯f) 2http://www.phramer.org/ – Java-based open-source phrase based SMT system 3http://www.isi.edu/licensed-sw/carmel/ 4http://www.speech.sri.com/projects/srilm/ 5http://www.iccs.inf.ed.ac.uk/∼pkoehn/training.tgz 150 • lexical weighting (Koehn et al., 2003): lex( ¯f|¯e, a) = n � i=1 1 |{j|(i, j) ∈ a}| � ∀(i,j)∈a w(fi|ej) lex(¯e| ¯f, a) = m � j=1 1 |{i|(i, j) ∈ a}| � ∀(i,j)∈a w(ej|fi) • phrase penalty: τ( ¯f|¯e) = e; log(τ( ¯f|¯e)) = …

Content truncated to 500 words