Perplexity Minimization for Translation Model Domain Adaptation in Statistical Machine Translation

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 539–549, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Perplexity Minimization for Translation Model Domain Adaptation in Statistical Machine Translation Rico Sennrich Institute of Computational Linguistics University of Zurich Binzmühlestr. 14 CH-8050 Zürich sennrich@cl.uzh.ch Abstract We investigate the problem of domain adaptation for parallel data in Statistical Machine Translation (SMT). While tech- niques for domain adaptation of monolin- gual data can be borrowed for parallel data, we explore conceptual differences between translation model and language model do- main adaptation and their effect on per- formance, such as the fact that translation models typically consist of several features that have different characteristics and can be optimized separately. We also explore adapting multiple (4–10) data sets with no a priori distinction between in-domain and out-of-domain data except for an in-domain development set. 1 Introduction The increasing availability of parallel corpora from various sources, welcome as it may be, leads to new challenges when building a statis- tical machine translation system for a specific domain. The task of determining which par- allel texts should be included for training, and which ones hurt translation performance, is te- dious when performed through trial-and-error. Alternatively, methods for a weighted combina- tion exist, but there is conflicting evidence as to which approach works best, and the issue of de- termining weights is not adequately resolved. The picture looks better in language mod- elling, where model interpolation through per- plexity minimization has become a widespread method of domain adaptation. We investigate the applicability of this method for translation mod- els, and discuss possible applications. We move the focus away from a binary com- bination of in-domain and out-of-domain data. If we can scale up the number of models whose con- tributions we weight, this reduces the need for a priori knowledge about the fitness1 of each poten- tial training text, and opens new research oppor- tunities, for instance experiments with clustered training data. 2 Domain Adaptation for Translation Models To motivate efforts in domain adaptation, let us review why additional training data can improve, but also decrease translation quality. Adding more training data to a translation sys- tem is easy to motivate through the data sparse- ness problem. Koehn and Knight (2001) show that translation quality correlates strongly with how often a word occurs in the training corpus. Rare words or phrases pose a problem in sev- eral stages of MT modelling, from word align- ment to the computation of translation probabil- ities through Maximum Likelihood Estimation. Unknown words are typically copied verbatim to the target text, which may be a good strategy for named entities, but is often wrong otherwise. In general, more data allows for a better word align- ment, a better estimation of translation probabili- ties, and for the consideration of more context (in phrase-based or syntactic SMT). A second effect of additional data is not nec- essarily positive. Translations are inherently am- biguous, and …

Content truncated to 500 words