Domain Adaptation for Statistical Machine Translation with Monolingual Resources

Proceedings of the Fourth Workshop on Statistical Machine Translation , pages 182–189, Athens, Greece, 30 March – 31 March 2009. c⃝2009 Association for Computational Linguistics Domain Adaptation for Statistical Machine Translation with Monolingual Resources Nicola Bertoldi Marcello Federico FBK-irst - Ricerca Scientifica e Tecnologica Via Sommarive 18, Povo (TN), Italy {bertoldi, federico}@fbk.eu Abstract Domain adaptation has recently gained interest in statistical machine translation to cope with the performance drop ob- served when testing conditions deviate from training conditions. The basic idea is that in-domain training data can be ex- ploited to adapt all components of an al- ready developed system. Previous work showed small performance gains by adapt- ing from limited in-domain bilingual data. Here, we aim instead at significant per- formance gains by exploiting large but cheap monolingual in-domain data, either in the source or in the target language. We propose to synthesize a bilingual cor- pus by translating the monolingual adap- tation data into the counterpart language. Investigations were conducted on a state- of-the-art phrase-based system trained on the Spanish–English part of the UN cor- pus, and adapted on the corresponding Europarl data. Translation, re-ordering, and language models were estimated after translating in-domain texts with the base- line. By optimizing the interpolation of these models on a development set the BLEU score was improved from 22.60% to 28.10% on a test set. 1 Introduction A well-known problem of Statistical Machine Translation (SMT) is that performance quickly de- grades as soon as testing conditions deviate from training conditions. The very simple reason is that the underlying statistical models always tend to closely approximate the empirical distributions of the training data, which typically consist of bilin- gual texts and monolingual target-language texts. The former provide a means to learn likely trans- lations pairs, the latter to form correct sentences with translated words. Besides the general diffi- culties of language translation, which we do not consider here, there are two aspects that make machine learning of this task particularly hard. First, human language has intrinsically very sparse statistics at the surface level, hence gaining com- plete knowledge on translation phrase pairs or tar- get language n-grams is almost impractical. Sec- ond, language is highly variable with respect to several dimensions, style, genre, domain, topics, etc. Even apparently small differences in domain might result in significant deviations in the un- derlying statistical models. While data sparseness corroborates the need of large language samples in SMT, linguistic variability would indeed suggest to consider many alternative data sources as well. By rephrasing a famous saying we could say that “no data is better than more and assorted data”. The availability of language resources for SMT has dramatically increased over the last decade, at least for a subset of relevant languages and es- pecially for what concerns monolingual corpora. Unfortunately, the increase in quantity has not gone in parallel with an increase in assortment, es- pecially for what concerns the most valuable re- source, that is bilingual corpora. Large parallel data available to the research …

Content truncated to 500 words