Discriminative Instance Weighting for Domain Adaptation in Statistical Machine Translation

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 451–459, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Crown in Right of Canada. Discriminative Instance Weighting for Domain Adaptation in Statistical Machine Translation George Foster and Cyril Goutte and Roland Kuhn National Research Council Canada 283 Alexandre-Tach´e Blvd Gatineau, QC J8X 3X7 first.last@nrc.gc.ca Abstract We describe a new approach to SMT adapta- tion that weights out-of-domain phrase pairs according to their relevance to the target do- main, determined by both how similar to it they appear to be, and whether they belong to general language or not. This extends previ- ous work on discriminative weighting by us- ing a finer granularity, focusing on the prop- erties of instances rather than corpus com- ponents, and using a simpler training proce- dure. We incorporate instance weighting into a mixture-model framework, and find that it yields consistent improvements over a wide range of baselines. 1 Introduction Domain adaptation is a common concern when op- timizing empirical NLP applications. Even when there is training data available in the domain of inter- est, there is often additional data from other domains that could in principle be used to improve perfor- mance. Realizing gains in practice can be challeng- ing, however, particularly when the target domain is distant from the background data. For developers of Statistical Machine Translation (SMT) systems, an additional complication is the heterogeneous na- ture of SMT components (word-alignment model, language model, translation model, etc.), which pre- cludes a single universal approach to adaptation. In this paper we study the problem of us- ing a parallel corpus from a background domain (OUT) to improve performance on a target do- main (IN) for which a smaller amount of parallel training material—though adequate for reasonable performance—is also available. This is a standard adaptation problem for SMT. It is difficult when IN and OUT are dissimilar, as they are in the cases we study. For simplicity, we assume that OUT is ho- mogeneous. The techniques we develop can be ex- tended in a relatively straightforward manner to the more general case when OUT consists of multiple sub-domains. There is a fairly large body of work on SMT adaptation. We introduce several new ideas. First, we aim to explicitly characterize examples from OUT as belonging to general language or not. Pre- vious approaches have tried to find examples that are similar to the target domain. This is less ef- fective in our setting, where IN and OUT are dis- parate. The idea of distinguishing between general and domain-specific examples is due to Daum´e and Marcu (2006), who used a maximum-entropy model with latent variables to capture the degree of speci- ficity. Daum´e (2007) applies a related idea in a simpler way, by splitting features into general and domain-specific versions. This highly effective ap- proach is not directly applicable to the multinomial models used for core SMT components, which have no natural method for combining split features, so we rely on an instance-weighting approach (Jiang …

Content truncated to 500 words