Extracting In-domain Training Corpora for Neural Machine Translation Using Data Selection Methods

Proceedings of the Third Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 224–231 Belgium, Brussels, October 31 - Novermber 1, 2018. c⃝2018 Association for Computational Linguistics https://doi.org/10.18653/v1/W18-64023 Extracting In-domain Training Corpora for Neural Machine Translation Using Data Selection Methods Catarina Cruz Silva Unbabel R. Visc. de Santar´em 67B, 1000-286 Lisboa, Portugal catarina@unbabel.com Chao-Hong Liu, Alberto Poncelas, Andy Way ADAPT Centre, School of Computing, Dublin City University Dublin 9, Ireland {chaohong.liu, alberto.poncelas, andy.way}@adaptcentre.ie Abstract Data selection is a process used in select- ing a subset of parallel data for the training of machine translation (MT) systems, so that 1) resources for training might be reduced, 2) trained models could perform better than those trained with the whole corpus, and/or 3) trained models are more tailored to specific do- mains. It has been shown that for statistical MT (SMT), the use of data selection helps im- prove the MT performance significantly. In this study, we reviewed three data selection approaches for MT, namely Term Frequency– Inverse Document Frequency, Cross-Entropy Difference and Feature Decay Algorithm, and conducted experiments on Neural Machine Translation (NMT) with the selected data us- ing the three approaches. The results showed that for NMT systems, using data selection also improved the performance, though the gain is not as much as for SMT systems. 1 Introduction Data selection is a technology used to improve Machine Translation (MT) performance by choos- ing a subset of the corpus for the training of MT systems (Chen et al., 2016). There are additional benefits using subsets instead of the whole corpus for MT training. Firstly, the training time could be reduced significantly. In some application sce- narios, a much shorter training time would be very useful. Secondly, we could select data with the aim to make trained systems perform well for spe- cific domains. In MT, models built with in-domain data perform better, as the vocabulary and sentence structures used in one domain (e.g. legal) differs from another unrelated domain (e.g. biotechnol- ogy). There are several studies on data selection meth- ods for SMT, showing good improvements over the baselines in which the whole corpora were used for training (Chen et al., 2016). A popular data selection method is cross-entropy difference (CED) (Moore and Lewis, 2010). In particular its bilingual variant (Axelrod et al., 2011) showed a positive im- pact of data selection for MT. Term Frequency-Inverse Document Frequency (TF-IDF) (Salton and Yang, 1973) has also been used as a baseline data selection method in the liter- ature. Data selection with cleaning was proposed to improve the robustness of training with divergent sentences (Carpuat et al., 2017). Feature Decay Algorithms (FDA) are data se- lection methods that try to extract the subset of sentences by which the coverage of target language features is maximized (Bic¸ici and Yuret, 2011). It has been used to select sentences from parallel data for SMT and NMT (Poncelas et al., 2018) in order to obtain a subset of data that is more tailored to a given test …

Content truncated to 500 words