Parameter Optimization for Statistical Machine Translation: It Pays to Learn from Hard Examples

Proceedings of Recent Advances in Natural Language Processing, pages 504–510, Hissar, Bulgaria, 7-13 September 2013. Parameter Optimization for Statistical Machine Translation: It Pays to Learn from Hard Examples Preslav Nakov, Fahad Al Obaidli, Francisco Guzm´an and Stephan Vogel Qatar Computing Research Institute, Qatar Foundation Tornado Tower, floor 10, PO box 5825 Doha, Qatar {pnakov,faalobaidli,fherrera,svogel}@qf.org.qa Abstract Research on statistical machine transla- tion has focused on particular translation directions, typically with English as the target language, e.g., from Arabic to En- glish. When we reverse the translation di- rection, the multiple reference translations turn into multiple possible inputs, which offers both challenges and opportunities. We propose and evaluate several strategies for making use of these multiple inputs: (a) select one of the datasets, (b) select the best input for each sentence, and (c) syn- thesize an input for each sentence by fus- ing the available inputs. Surprisingly, we find out that it is best to tune on the hardest available input, not on the one that yields the highest BLEU score. This finding has implications on how to pick good transla- tors and how to select useful data for pa- rameter optimization in SMT. 1 Introduction Nowadays, statistical machine translation (SMT) systems are data-driven, and thus critically depend on the available resources for training, tuning and evaluation. These resources are hard to obtain, which has limited research to a small number of language pairs for which biligual sentence-aligned parallel corpora, called bitexts, are available. What is often not realized is that SMT research has further been restricted to only some transla- tion directions, e.g., those of interest to evaluation campaigns such as NIST and IWSLT or to funding agencies such as DARPA. This is because stable SMT evaluation requires multiple reference trans- lations for the target language. Such multiple ref- erences are often available for the English (target) side of the tuning and the evaluation dataset, but not for the source language, e.g., Arabic, Chinese. Reversing the translation direction yields (i) a single reference translation and (ii) multiple ver- sions for each tuning/testing input sentence. There is little we can do about (i),1 but (ii) offers inter- esting opportunities for tuning and evaluation. Below we focus on the question of how to make best use of the multiple available inputs at tuning time. We propose and evaluate several strategies for making use of these multiple inputs: (a) select one of the datasets, (b) select the best input for each sentence, and (c) synthesize an input for each sentence by fusing the available inputs. 2 Related Work One relevant line of research is on multi-source translation, which generates a single translation given multiple versions of the input. This line was started by Och and Ney (2001), who translated the different inputs in isolation and then selected one of them. It has been further extended with vari- ous strategies for generating a consensus transla- tion by combining either the inputs (Schroeder et al., 2009) or the outputs (Matusov et al., 2006) of the SMT …

Content truncated to 500 words