Monolingual Marginal Matching for Translation Model Adaptation

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1077–1088, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Monolingual Marginal Matching for Translation Model Adaptation Ann Irvine Johns Hopkins University anni@jhu.edu Chris Quirk Microsoft Research chrisq@microsoft.com Hal Daum´e III University of Maryland me@hal3.name Abstract When using a machine translation (MT) model trained on OLD-domain parallel data to translate NEW-domain text, one major chal- lenge is the large number of out-of-vocabulary (OOV) and new-translation-sense words. We present a method to identify new translations of both known and unknown source language words that uses NEW-domain comparable doc- ument pairs. Starting with a joint distribution of source-target word pairs derived from the OLD-domain parallel corpus, our method re- covers a new joint distribution that matches the marginal distributions of the NEW-domain comparable document pairs, while minimiz- ing the divergence from the OLD-domain dis- tribution. Adding learned translations to our French-English MT model results in gains of about 2 BLEU points over strong baselines. 1 Introduction When a statistical machine translation (SMT) model trained on OLD-domain (e.g. parliamentary proceed- ings) parallel text is used to translate text in a NEW- domain (e.g. medical or scientific), performance de- grades drastically. One of the major causes is the large number of NEW-domain words that are out-of- vocabulary (OOV) with respect to the OLD-domain text. Figure 1 shows the OOV rate for text in several NEW-domains, with respect to OLD-domain parlia- mentary proceedings. Even more challenging are the difficult-to-detect new-translation-sense (NTS) words: French words that are present in both the OLD and NEW domains but that are translated dif- ferently in each domain. For example, the French Parliament Subtitles Medical Science Percent of Word Types that are OOV 0 10 20 30 40 50 60 3.95 27.03 41.42 50.93 Figure 1: Percent of test set word types by domain that are OOV with respect to five million tokens of OLD- domain French parliamentary proceedings data. word enceinte is mostly translated in parliamentary proceedings as place, house, or chamber; in medical text, the translation is mostly pregnant; in scientific text, enclosures. One potential remedy is to collect parallel data in the NEW-domain, from which we can train a new SMT model. Smith et al. (2010), for example, mine parallel text from comparable corpora. Parallel sen- tences are informative but also rare: in the data re- leased by Smith et al. (2010), only 21% of the for- eign sentences have a near-parallel counterpart in the English article.1 Furthermore, these sentences do not capture all terms. In that same dataset, we find that on average only 20% of foreign and 28% of En- glish word types in a given article are represented in the parallel sentence pairs. In this work, we seek to learn a joint distribu- 1Only 12% of sentences from generally longer English arti- cles have a near-parallel counterpart in the foreign language. 1077 tion of translation probabilities over all source and target word pairs in the NEW-domain. We begin with …

Content truncated to 500 words