Adapting to All Domains at Once: Rewarding Domain Invariance in SMT

Adapting to All Domains at Once: Rewarding Domain Invariance in SMT Hoang Cuong and Khalil Sima’an and Ivan Titov Institute for Logic, Language and Computation University of Amsterdam Science Park 107, 1098 XG Amsterdam, The Netherlands {c.hoang,k.simaan,titov}@uva.nl Abstract Existing work on domain adaptation for statis- tical machine translation has consistently as- sumed access to a small sample from the test distribution (target domain) at training time. In practice, however, the target domain may not be known at training time or it may change to match user needs. In such situations, it is natural to push the system to make safer choices, giving higher preference to domain- invariant translations, which work well across domains, over risky domain-specific alterna- tives. We encode this intuition by (1) in- ducing latent subdomains from the training data only; (2) introducing features which mea- sure how specialized phrases are to individual induced sub-domains; (3) estimating feature weights on out-of-domain data (rather than on the target domain). We conduct experiments on three language pairs and a number of differ- ent domains. We observe consistent improve- ments over a baseline which does not explic- itly reward domain invariance. 1 Introduction Mismatch in phrase translation distributions be- tween test data (target domain) and train data is known to harm performance of statistical transla- tion systems (Irvine et al., 2013; Carpuat et al., 2014). Domain-adaptation methods (Foster et al., 2010; Bisazza et al., 2011; Sennrich, 2012b; Raz- mara et al., 2012; Sennrich et al., 2013; Haddow, 2013; Joty et al., 2015) aim to specialize a system estimated on out-of-domain training data to a target domain represented by a small data sample. In prac- tice, however, the target domain may not be known at training time or it may change over time depend- ing on user needs. In this work we address exactly the setting where we have a domain-agnostic system but we have no access to any samples from the tar- get domain at training time. This is an important and challenging setting which, as far as we are aware, has not yet received attention in the literature. When the target domain is unknown at training time, the system could be trained to make safer choices, preferring translations which are likely to work across different domains. For example, when translating from English to Russian, the most natural translation for the word ‘code’ would be highly de- pendent on the domain (and the corresponding word sense). The Russian words ‘xifr’, ‘zakon’ or ‘programma’ would perhaps be optimal choices if we consider cryptography, legal and software devel- opment domains, respectively. However, the transla- tion ‘kod’ is also acceptable across all these domains and, as such, would be a safer choice when the tar- get domain is unknown. Note that such a transla- tion may not be the most frequent overall and, con- sequently, might not be proposed by a standard (i.e., domain-agnostic) phrase-based translation system. In order to encode preference for domain- invariant translations, we introduce a measure which quantifies how likely …

Content truncated to 500 words