Semi-supervised Convolutional Networks for Translation Adaptation with Tiny Amount of In-domain Data

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 314–323, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Semi-supervised Convolutional Networks for Translation Adaptation with Tiny Amount of In-domain Data Boxing Chen National Research Council Canada 1200 Montreal Road Ottawa, ON, Canada Boxing.Chen@nrc-cnrc.gc.ca Fei Huang Facebook 770 Broadway New York, NY, USA feihuang@fb.com Abstract In this paper, we propose a method which uses semi-supervised convolutional neu- ral networks (CNNs) to select in-domain training data for statistical machine trans- lation. This approach is particularly effec- tive when only tiny amounts of in-domain data are available. The in-domain data and randomly sampled general-domain data are used to train a data selection model with semi-supervised CNN, then this model computes domain relevance scores for all the sentences in the general- domain data set. The sentence pairs with top scores are selected to train the sys- tem. We carry out experiments on 4 lan- guage directions with three test domains. Compared with strong baseline systems trained with large amount of data, this method can improve the performance up to 3.1 BLEU. Its performances are signifi- cant better than three state-of-the-art lan- guage model based data selection meth- ods. We also show that the in-domain data used to train the selection model could be as few as 100 sentences, which makes fine- grained topic-dependent translation adap- tation possible. 1 Introduction Statistical machine translation (SMT) systems are trained on bilingual parallel and monolingual data. The training corpora typically come from different sources, and vary across topics, genres, dialects, authors’ written styles, etc., which are usually re- ferred as “general domain” training data. Here the word “domain” is often used to indicate some combination of all above and other possible hid- den factors (Chen et al., 2013). At run time, the content to be translated may come from a different domain. Due to the mismatch in “domains”, it is possible to achieve better performance by adapting the SMT system to the test domain (in-domain). However, manually creating training data to match the test domain is not a preferred solu- tion, because 1) sometimes the test domain is not known when training the model, and it could change from sentence to sentence; 2) even if the test domain is pre-determined, the resources re- quired and slow turnaround in data collection pro- cess will still delay the system development pro- cess. Therefore, training data selection is widely used for domain adaptation in statistical machine trans- lation (Zhao et al., 2004; L¨u et al., 2007; Ya- suda et al., 2008; Moore and Lewis, 2010; Ax- elrod et al., 2011; Duh et al., 2013; Axelrod et al., 2015). Data selection techniques select mono- lingual or bilingual data that are similar to the in- domain seed data based on some criteria, which are incorporated into the training data. The most successful data selection approaches (Moore and Lewis, 2010; Axelrod et al., 2011) train n-gram language models on in-domain text to select sim- ilar sentences from the …

Content truncated to 500 words