Discriminative Weighted Alignment Matrices For Statistical Machine Translation

Discriminative Weighted Alignment Matrices For Statistical Machine Translation Nadi Tomeh and Alexandre Allauzen and François Yvon LIMSI-CNRS and Université Paris Sud BP 133, 91403 Orsay {nadi,allauzen,yvon}@limsi.fr Abstract In extant phrase-based statistical machine translation (SMT) systems, the transla- tion model relies on word-to-word align- ments, which serve as constraints for the subsequent heuristic extraction and scor- ing processes. Word alignments are usu- ally inferred in a probabilistic framework; yet, only one single best alignment is re- tained, as if alignments were deterministi- cally produced. In this paper, we explore ways to take into account the entire align- ment matrix, where each alignment link is scored by its probability. By compari- son with previous attempts, we use an ex- ponential model to compute these proba- bilities, which enables us to achieve sig- nificant improvements on the NIST MT’09 Arabic-English translation task. 1 Introduction In Phrase-Based SMT systems, a source sentence is translated by concatenating translation options, selected from an inventory called the phrase table. Building this inventory from a parallel corpus con- stitute the translation model training phase, which is usually performed in two main steps. 1) For each training sentence pair, a set of source-target phrase-pairs, that are translations of one another, are first extracted. 2) Phrase pairs accumulated over the entire training corpus are collected and scored using relative frequencies estimates. The collection of phrase-pairs and their scores consti- tutes the translation model. During the extraction step, we would like to use a phrase alignment model that enables the compu- c⃝ 2011 European Association for Machine Translation. tation of corpus level statistics related to the joint segmentation and alignment of source and target sentences. Unfortunately, generative models de- signed for this purpose (Marcu and Wong, 2002; Birch et al., 2006) fail to deliver good performance due to three key difficulties (DeNero et al., 2006). First, exploring the whole space of phrase- to-phrase alignment is intractable, which makes phrase alignment a NP-hard (DeNero and Klein, 2008) problem. Second, including a latent seg- mentation variable in the model increases the risk of overfitting during EM training. Third, spuri- ous segmentation ambiguity tends to populate the phrase table with more entries, each having too few translation options. A practical solution is to re- configure the phrase alignment problem in terms of words instead of phrases: a fixed segmentation, based on word boundaries, is used, and the result- ing model is simpler to train using EM. Then, for each word-aligned sentence in the training corpus, an additional step is required to identify the set of phrase-pairs to be extracted. A heuristic which extracts phrase-pairs that are consistent with the Viterbi word alignment is widely used in practice. During the scoring step, relative frequencies computed on the training corpus are used to as- sess each extracted phrase pair. Additional scores, based on lexical probabilities are also used so as to smooth the scores of rare phrase-pairs. The training of the translation model is thus de- composed as a modular pipeline the components of which can …

Content truncated to 500 words