Domain Adaptation via Pseudo In-Domain Data Selection

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 355–362, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Domain Adaptation via Pseudo In-Domain Data Selection Amittai Axelrod University of Washington Seattle, WA 98105 amittai@uw.edu Xiaodong He Microsoft Research Redmond, WA 98052 xiaohe@microsoft.com Jianfeng Gao Microsoft Research Redmond, WA 98052 jfgao@microsoft.com Abstract We explore efficient domain adaptation for the task of statistical machine translation based on extracting sentences from a large general- domain parallel corpus that are most relevant to the target domain. These sentences may be selected with simple cross-entropy based methods, of which we present three. As these sentences are not themselves identical to the in-domain data, we call them pseudo in-domain subcorpora. These subcorpora – 1% the size of the original – can then used to train small domain-adapted Statistical Ma- chine Translation (SMT) systems which out- perform systems trained on the entire corpus. Performance is further improved when we use these domain-adapted models in combination with a true in-domain model. The results show that more training data is not always better, and that best results are attained via proper domain-relevant data selection, as well as combining in- and general-domain systems during decoding. 1 Introduction Statistical Machine Translation (SMT) system per- formance is dependent on the quantity and quality of available training data. The conventional wisdom is that more data is better; the larger the training cor- pus, the more accurate the model can be. The trouble is that – except for the few all-purpose SMT systems – there is never enough training data that is directly relevant to the translation task at hand. Even if there is no formal genre for the text to be translated, any coherent translation task will have its own argot, vocabulary or stylistic prefer- ences, such that the corpus characteristics will nec- essarily deviate from any all-encompassing model of language. For this reason, one would prefer to use more in-domain data for training. This would em- pirically provide more accurate lexical probabilities, and thus better target the task at hand. However, par- allel in-domain data is usually hard to find1, and so performance is assumed to be limited by the quan- tity of domain-specific training data used to build the model. Additional parallel data can be readily ac- quired, but at the cost of specificity: either the data is entirely unrelated to the task at hand, or the data is from a broad enough pool of topics and styles, such as the web, that any use this corpus may provide is due to its size, and not its relevance. The task of domain adaptation is to translate a text in a particular (target) domain for which only a small amount of training data is available, using an MT system trained on a larger set of data that is not re- stricted to the target domain. We call this larger set of data a general-domain corpus, in lieu of the stan- dard yet slightly misleading out-of-domain …

Content truncated to 500 words