Length-Incremental Phrase Training for SMT

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 309–319, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics Length-incremental Phrase Training for SMT Joern Wuebker and Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen, Germany {wuebker,ney}@cs.rwth-aachen.de Abstract We present an iterative technique to gener- ate phrase tables for SMT, which is based on force-aligning the training data with a modified translation decoder. Differ- ent from previous work, we completely avoid the use of a word alignment or phrase extraction heuristics, moving to- wards a more principled phrase generation and probability estimation. During train- ing, we allow the decoder to generate new phrases on-the-fly and increment the max- imum phrase length in each iteration. Ex- periments are carried out on the IWSLT 2011 Arabic-English task, where we are able to reach moderate improvements on a state-of-the-art baseline with our training method. The resulting phrase table shows only a small overlap with the heuristically extracted one, which demonstrates the re- strictiveness of limiting phrase selection by a word alignment or heuristics. By interpolating the heuristic and the trained phrase table, we can improve over the baseline by 0.5% BLEU and 0.5% TER. 1 Introduction Most state-of-the-art SMT systems get the statis- tics from which the different component models are estimated via heuristics using a Viterbi word alignment. The word alignment is usually gener- ated with tools like GIZA++ (Och and Ney, 2003), that apply the EM algorithm to estimate the align- ment with the HMM or IBM-4 translation mod- els. This is also the case for the phrases or rules which serve as translation units for the decoder. All phrases that do not violate the word alignment are extracted and their probabilities are estimated as relative frequencies (Koehn et al., 2003). A number of different approaches have tried to do away with the heuristics and close this gap be- tween the phrase table generation and translation decoding. However, most of these approaches ei- ther fail to achieve state-of-the-art performance or still make use of the word alignment or the ex- traction heuristics, e.g. as a prior in discriminative training or to initialize a generative or generatively inspired training procedure and are thus biased by their weaknesses. Here, we aim at moving towards the ideal situation, where a unified framework in- duces the phrases based on the same models as in decoding. We train the phrase table without using a word alignment or the extraction heuristics. Different from previous work, we are able to generate all possible phrase pairs on-the-fly during the train- ing procedure. A further advantage of our pro- posed algorithm is that we use basically the same beam search as in translation. This makes it easy to re-implement by modifying any translation de- coder, and makes sure that training and translation are consistent. In principle, we apply the forced decoding approach described in (Wuebker et al., 2010) with cross-validation to prevent over-fitting, but we initialize the phrase table with IBM-1 lex- ical …

Content truncated to 500 words