Vector Space Model for Adaptation in Statistical Machine Translation

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1285–1293, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Vector Space Model for Adaptation in Statistical Machine Translation Boxing Chen, Roland Kuhn and George Foster National Research Council Canada first.last@nrc-cnrc.gc.ca Abstract This paper proposes a new approach to domain adaptation in statistical machine translation (SMT) based on a vector space model (VSM). The general idea is first to create a vector profile for the in-domain development (“dev”) set. This profile might, for instance, be a vector with a di- mensionality equal to the number of train- ing subcorpora; each entry in the vector re- flects the contribution of a particular sub- corpus to all the phrase pairs that can be extracted from the dev set. Then, for each phrase pair extracted from the train- ing data, we create a vector with features defined in the same way, and calculate its similarity score with the vector represent- ing the dev set. Thus, we obtain a de- coding feature whose value represents the phrase pair’s closeness to the dev. This is a simple, computationally cheap form of instance weighting for phrase pairs. Ex- periments on large scale NIST evaluation data show improvements over strong base- lines: +1.8 BLEU on Arabic to English and +1.4 BLEU on Chinese to English over a non-adapted baseline, and signifi- cant improvements in most circumstances over baselines with linear mixture model adaptation. An informal analysis suggests that VSM adaptation may help in making a good choice among words with the same meaning, on the basis of style and genre. 1 Introduction The translation models of a statistical machine translation (SMT) system are trained on parallel data. Usage of language and therefore the best translation practice differs widely across genres, topics, and dialects, and even depends on a partic- ular author’s or publication’s style; the word “do- main” is often used to indicate a particular combi- nation of all these factors. Unless there is a per- fect match between the training data domain and the (test) domain in which the SMT system will be used, one can often get better performance by adapting the system to the test domain. Domain adaptation is an active topic in the nat- ural language processing (NLP) research commu- nity. Its application to SMT systems has recently received considerable attention. Approaches that have been tried for SMT model adaptation include mixture models, transductive learning, data selec- tion, instance weighting, and phrase sense disam- biguation, etc. Research on mixture models has considered both linear and log-linear mixtures. Both were studied in (Foster and Kuhn, 2007), which con- cluded that the best approach was to combine sub- models of the same type (for instance, several different TMs or several different LMs) linearly, while combining models of different types (for in- stance, a mixture TM with a mixture LM) log- linearly. (Koehn and Schroeder, 2007), instead, opted for combining the sub-models directly in the SMT log-linear framework. In transductive learning, an …

Content truncated to 500 words