Multi-Domain Adaptation for SMT Using Multi-Task Learning

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1055–1065, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Multi-domain Adaptation for SMT Using Multi-task Learning∗ Lei Cui1, Xilun Chen2, Dongdong Zhang3, Shujie Liu3, Mu Li3, and Ming Zhou3 1Harbin Institute of Technology, Harbin, P.R. China leicui@hit.edu.cn 2Cornell University, Ithaca, NY, U.S. xlchen@cs.cornell.edu 3Microsoft Research Asia, Beijing, P.R. China {dozhang,shujliu,muli,mingzhou}@microsoft.com Abstract Domain adaptation for SMT usually adapts models to an individual specific domain. However, it often lacks some correlation among different domains where common knowledge could be shared to improve the overall translation quality. In this paper, we propose a novel multi-domain adaptation ap- proach for SMT using Multi-Task Learning (MTL), with in-domain models tailored for each specific domain and a general-domain model shared by different domains. The pa- rameters of these models are tuned jointly via MTL so that they can learn general knowledge more accurately and exploit domain knowl- edge better. Our experiments on a large- scale English-to-Chinese translation task val- idate that the MTL-based adaptation approach significantly and consistently improves the translation quality compared to a non-adapted baseline. Furthermore, it also outperforms the individual adaptation of each specific domain. 1 Introduction Domain adaptation is an active topic in statisti- cal machine learning and aims to alleviate the do- main mismatch between training and testing data. Like many machine learning tasks, Statistical Ma- chine Translation (SMT) assumes that the data dis- tributions of training and testing domains are sim- ilar. However, this assumption does not hold for real world SMT systems since training data for SMT models may come from a variety of domains. The translation quality is often unsatisfactory when ∗This work was done while the first and second authors were visiting Microsoft Research Asia. translating texts from a specific domain using a gen- eral model that is trained over a hotchpotch of bilin- gual corpora. Therefore, domain adaptation is cru- cial for SMT systems to achieve better performance. Previous research on domain adaptation for SMT includes data selection and weighting (Eck et al., 2004; L¨u et al., 2007; Foster et al., 2010; Moore and Lewis, 2010; Axelrod et al., 2011), mixture mod- els (Foster and Kuhn, 2007; Koehn and Schroeder, 2007; Sennrich, 2012; Razmara et al., 2012), and semi-supervised transductive learning (Ueffing et al., 2007), etc. Most of these methods adapt SMT models to a specific domain according to testing data and have achieved good performance. It is natural that real world SMT systems should adapt the mod- els to multiple domains because the input may be heterogeneous, so that the overall translation qual- ity can be improved. Although we can easily ap- ply these methods to multiple domains individually, it is difficult to use the common knowledge across different domains. To leverage the common knowl- edge, we need to devise a multi-domain adaptation approach that jointly adapts the SMT models. Multi-domain adaptation has been proved quite effective in sentiment analysis (Dredze and Cram- mer, 2008) and web ranking (Chapelle …

Content truncated to 500 words