Mixture-Modeling with Unsupervised Clusters for Domain Adaptation in Statistical Machine Translation

Mixture-Modeling with Unsupervised Clusters for Domain Adaptation in Statistical Machine Translation Rico Sennrich Institute of Computational Linguistics University of Zurich Binzm¨uhlestr. 14 CH-8050 Z¨urich sennrich@cl.uzh.ch Abstract In Statistical Machine Translation, in- domain and out-of-domain training data are not always clearly delineated. This paper investigates how we can still use mixture-modeling techniques for domain adaptation in such cases. We apply un- supervised clustering methods to split the original training set, and then use mixture- modeling techniques to build a model adapted to a given target domain. We show that this approach improves performance over an unadapted baseline, and several al- ternative domain adaptation methods. 1 Introduction As the availability of parallel data for Statistical Machine Translation (SMT) increases, new op- portunities and challenges for domain adaptation arise. Some corpora may contain text from a variety of domains, especially if they are built from heterogeneous resources such as crawled web pages. Many domain adaptation techniques do not operate on a single text, but require multiple mod- els which are then mixed. We investigate domain adaptation in a scenario where we have a known target domain, including development and test data from this domain, but where there is only a single heterogeneous train- ing corpus. While this training corpus does con- tain in-domain data, we assume that we have no supervised means of extracting it. Our basic approach is divided into two steps. Firstly, we perform unsupervised clustering on the parallel training data to obtain a given number of clusters. Secondly, we apply domain adaptation c⃝ 2012 European Association for Machine Translation. algorithms to compute a model from these clusters that is adapted to the development set. 2 Related Work The general idea in domain adaptation is to obtain models that are specifically optimized for best per- formance in one domain, with a potentially nega- tive effect on its performance for other domains. The classical domain adaptation scenario consists of a (small) in-domain corpus, a (large) out-of- domain corpus, and in-domain development and test sets. Mixture-modeling approaches such as (Koehn and Schroeder, 2007; Foster and Kuhn, 2007; Sennrich, 2012) fall into this category. We will here give an overview of adaptation techniques that assume less prior knowledge about the training set and/or target domains. Yamamoto and Sumita (2008) operate without any predetermined domains, and without assum- ing that either the training or the test data is ho- mogeneous. They cluster the training text into k clusters, and use unsupervised domain selection to translate each test set sentence by a cluster-specific model. Finch and Sumita (2008) distinguish between two classes of sentences: questions and declara- tives (i.e. non-questions). They split the training corpus automatically according to a simple rule (does the target sentence end with ’?’), and for decoding use a linear interpolation of the class- specific and a general model, the interpolation weight depending on the class membership of each sentence. Banerjee et al. (2010) focus on a scenario in which the domains of the training texts are known, whereas the test sets …

Content truncated to 500 words