Linear Mixture Models for Robust Machine Translation

Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 499–509, Baltimore, Maryland USA, June 26–27, 2014. c⃝2014 Association for Computational Linguistics Linear Mixture Models for Robust Machine Translation Marine Carpuat, Cyril Goutte and George Foster Multilingual Text Processing National Research Council Ottawa, ON K1A0R6, Canada firstname.lastname@nrc.ca Abstract As larger and more diverse parallel texts become available, how can we lever- age heterogeneous data to train robust machine translation systems that achieve good translation quality on various test domains? This challenge has been ad- dressed so far by repurposing techniques developed for domain adaptation, such as linear mixture models which combine estimates learned on homogeneous sub- domains. However, learning from large heterogeneous corpora is quite different from standard adaptation tasks with clear domain distinctions. In this paper, we show that linear mixture models can re- liably improve translation quality in very heterogeneous training conditions, even if the mixtures do not use any domain knowledge and attempt to learn generic models rather than adapt them to the tar- get domain. This surprising finding opens new perspectives for using mixture mod- els in machine translation beyond clear cut domain adaptation tasks. 1 Introduction While machine translation tasks used to be de- fined by drawing training and test data from a sin- gle well-defined domain, current systems have to deal with increasingly heterogeneous data, both at training and at test time. As larger and more di- verse parallel texts become available, how can we leverage heterogeneous data to train statistical ma- chine translation (SMT) systems that achieve good translation quality on various test domains? So far, this challenge has been addressed by re- purposing techniques developed for more clear-cut domain adaptation scenarios, such as linear mix- ture models (Koehn and Schroeder, 2007; Foster and Kuhn, 2007; Sennrich, 2012b). Instead of es- timating models on the whole training corpus at once, linear mixture models are built as follows: (1) partition the training corpus into homogeneous domain-based component, (2) train one model per component, (3) linearly mix models using weights learned to adapt to the test domain, (4) replace re- sulting model in translation system. In this paper, we aim to gain a better under- standing of the benefits of linear mixture models in heterogeneous data conditions, by examining key untested assumptions: • Should mixture component capture domain information? Previous work assumes that training data should be organized into do- mains. When manual domain distinctions are not available, previous work uses clustering approaches to approximate manual domain distinctions (Sennrich, 2012a). However, it is unclear whether it is necessary to use or mimic domain distinctions in order to define mixture components. • Mixture models are usually assumed to im- prove translation quality by giving more weight to parts of the training corpus that are more relevant to the test domain. Is this intu- ition still valid in our more complex hetero- geneous training conditions? If not, how do mixture models affect translation probability estimates? In order to answer these questions, we propose to study several variants …

Content truncated to 500 words