Combination of Neural Machine Translation Systems at WMT20

Proceedings of the 5th Conference on Machine Translation (WMT), pages 230–238 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 230 Combination of Neural Machine Translation Systems at WMT20 Benjamin Marie Raphael Rubino Atsushi Fujita National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japan {bmarie, raphael.rubino, atsushi.fujita}@nict.go.jp Abstract This paper presents neural machine trans- lation systems and their combination built for the WMT20 English↔Polish and Japanese→English translation tasks. We show that using a Transformer Big architecture, additional training data synthesized from monolingual data, and combining many NMT systems through n-best list reranking improve translation quality. However, while we observed such improvements on the validation data, we did not observe similar improvements on the test data. Our analysis reveals that the presence of translationese texts in the validation data led us to take decisions in building NMT systems that were not optimal to obtain the best results on the test data. 1 Introduction This paper describes the neural machine trans- lation (NMT) systems and their combination built for a participation of the National Insti- tute of Information and Communications Technol- ogy (NICT) in the WMT20 shared News Trans- lation Task.1 lation directions:We participated in three trans- Japanese→English (Ja→En), English→Polish (En→Pl), and Polish→English (Pl→En). All our systems are constrained, i.e., we used only the parallel and monolingual data pro- vided by the organizers to train and tune them, and validated/selected our best systems exclusively us- ing the official validation data provided by the or- ganizers. We trained NMT systems with several different frameworks and architectures, and com- bined them, for each translation direction, through n-best list reranking using informative features as proposed by Marie and Fujita (2018). This sim- ple combination method, associated with the ex- ploitation of large tagged back-translated mono- lingual data, improved BLEU scores on the official 1The team ID of our participation is “NICT Kyoto”. validation data provided by the organizers. How- ever, we did not observe these improvements on the test data for which our baseline systems re- mained the best. While we have rigorously se- lected our systems according to their performance on the validation data, the analysis of our results reveal how easily we would have been able to achieve BLEU scores among the best submissions by choosing/selecting our best systems according to their performance on the test data, as encour- aged by the WMT submission process (Section 2). The remainder of this paper is organized as follows. In Section 2, we briefly describe the WMT20 translation task. In Section 3, we intro- duce the data pre-processing and cleaning. In Sec- tion 4, we describe the details of our NMT sys- tems’ architectures and frameworks. In Section 5, we describe two different strategies that we used to augment the training parallel data of our systems: parallel data extraction from monolingual data and backward/forward translations. Then, the combi- nation of our NMT systems is described in Sec- tion 6. Empirical results produced with our sys- tems on the validation and test …

Content truncated to 500 words