Parametric Weighting of Parallel Data for Statistical Machine Translation

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1323–1331, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Parametric Weighting of Parallel Data for Statistical Machine Translation Kashif Shah, Lo¨ıc Barrault, Holger Schwenk LIUM, University of Le Mans Le Mans, France. FirstName.LastName@lium.univ-lemans.fr Abstract During the last years there is increas- ing interest in methods that perform some kind of weighting of heterogeneous paral- lel training data when building a statistical machine translation system. It was for in- stance observed that training data that is close to the period of the test data is more valuable than older data (Hardt and Elm- ing, 2010; Levenberg et al., 2010). In this paper we obtain such a weighting by resampling alignments using weights that decrease with the temporal distance of bi- texts to the test set. By these means, we can use all the available bitexts and still put an emphasis on the most recent one. The main idea of our approach is to use a parametric form or meta-weights for the weighting of the different parts of the bi- texts. This ensures that our approach has only few parameters to optimize. We re- port experimental results on the Europarl corpus, translating from French to En- glish and further verified it on the official WMT’11 task, translating from English to French. Our method achieves improve- ments of about 0.6 points BLEU on the test set with respect to a system trained on data without any weighting. 1 Introduction Statistical machine translation (SMT) systems are based on two types of resources: monolingual data to build a language model (LM) and bilingual data – also called bitexts – to train the translation model (TM). The parallel data often comes from differ- ent sources, e.g. Europarl, UN, in-domain data in limited amounts, data crawled from the Internet or even bitexts automatically extracted from compa- rable corpora. It seems obvious that the appropri- ateness and the usefulness of this parallel data for a particular translation task may vary quite a lot. Nevertheless, the standard procedure is to concate- nate all available parallel data, to perform word alignment using GIZA++ (Och and Ney, 2000) and to extract and score the phrase pairs by sim- ple relative frequency. Doing this, the parallel data is (wrongly) considered as one homogeneous pool of knowledge. We argue that the parallel data is quite inhomogeneous in many practical applica- tions with respect to several factors: • the data may come from different sources that are more or less relevant to the translation task (in-domain versus out-of-domain data). • more generally, the topic or genre of the data may be more or less relevant. • the data may be of different quality (carefully performed human translations versus auto- matically crawled and aligned data). • the recency of the data with respect to the task may have an influence. This is of interest in the news domain where named entities, etc change over time. There have been several attempts in …

Content truncated to 500 words