Tilde at WMT 2020: News Task Systems

Proceedings of the 5th Conference on Machine Translation (WMT), pages 175–180 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 175 Tilde at WMT 2020: News Task Systems Rihards Kriˇslauks†‡ and M¯arcis Pinnis†‡ †Tilde / Vienibas gatve 75A, Riga, Latvia ‡Faculty of Computing, University of Latvia / Rain¸a bulv. 19, Riga, Latvia {firstname.lastname}@tilde.lv Abstract This paper describes Tilde’s submission to the WMT2020 shared task on news translation for both directions of the English↔Polish lan- guage pair in both the constrained and the un- constrained tracks. We follow our submissions form the previous years and build our base- line systems to be morphologically motivated sub-word unit-based Transformer base models that we train using the Marian machine trans- lation toolkit. Additionally, we experiment with different parallel and monolingual data selection schemes, as well as sampled back- translation. Our final models are ensembles of Transformer base and Transformer big models which feature right-to-left re-ranking. 1 Introduction This year, we developed both constrained and un- constrained NMT systems for the English↔Polish language pair. We base our methods on the sub- missions of the previous years (Pinnis et al., 2017b, 2018, 2019) including methods for parallel data filtering from Pinnis (2018). Specifically, we lean on Pinnis (2018) and Junczys-Dowmunt (2018) for data selection and filtering, (Pinnis et al., 2017b) for morphologically motivated sub-word units and synthetic data generation, Edunov et al. (2018) for sampled back-translation and finally Morishita et al. (2018) for re-ranking with right-to-left models. We use the Marian toolkit (Junczys-Dowmunt et al., 2018) to train models of Transformer architecture (Vaswani et al., 2017). Although document level NMT as showcased by (Junczys-Dowmunt, 2019) have yielded promising results for the English-German language pair, we were not able to collect sufficient document level data for the English-Polish language pair. As a result, all our systems this year translate individual sentences. The paper is further structured as follows: Sec- tion 2 describes the data used to train our NMT systems, Section 3 describes our efforts to identify the best-performing recipes for training of our final systems, Section 5 summarises the results of our final systems, and Section 6 concludes the paper. 2 Data For training of the constrained NMT systems, we used data from the WMT 2020 shared task on news translation1. For unconstrained systems, we used data from the Tilde Data Library2. The 10 largest publicly available datasets that were used to train the unconstrained systems were Open Sub- titles from the Opus corpus (Tiedemann, 2016), ParaCrawl (Ban´on et al., 2020) (although it was discarded due to noise found in the corpus), DGT Translation Memories (Steinberger et al., 2012), Microsoft Translation and User Interface Strings Glossaries3 from multiple releases up to 2018, the Tilde MODEL corpus (Rozis and Skadin¸ˇs, 2017), WikiMatrix (Schwenk et al., 2019), Digital Corpus of the European Parliament (Hajlaoui et al., 2014), JRC-Acquis (Steinberger et al., 2006), Europarl (Koehn, 2005), and the QCRI Educational Domain Corpus (Abdelali et al., 2014). 2.1 Data Filtering and Pre-Processing First, we filtered data using Tilde’s parallel data …

Content truncated to 500 words