The University of Edinburgh’s submission to the German-to-English and English-to-German Tracks in the WMT 2020 News Translation and Zero-shot Translation Robustness Tasks

Proceedings of the 5th Conference on Machine Translation (WMT), pages 197–201 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 197 The University of Edinburgh’s submission to the German-to-English and English-to-German Tracks in the WMT 2020 News Translation and Zero-shot Translation Robustness Tasks Ulrich Germann University of Edinburgh, UK ugermann@inf.ed.ac.uk Abstract This paper describes the University of Ed- inburgh’s Submission of German ↔ En- glish systems to the WMT2020 Shared Tasks on News Translation and Zero-shot Robustness. 1 Introduction This paper describes the University of Edinburgh’s submission to the German-to-English and English- to-German tracks in the WMT 2020 News Trans- lation and Zero-shot Robust translation tasks. We built our systems in three stages, loosely follow- ing the procedure by Junczys-Dowmunt (2018b). All translation models mentioned in this paper were trained with the Marian toolkit (Junczys- Dowmunt et al., 2018). 2 3-stage Build Process We distinguish three types of training data pro- vided for the tasks, shown in Table 1. Table 1: Data available for training systems corpus sentence( pair)s High-quality parallel data Europarl ca. 1.79 M Rapid ca. 1.45 M News Commentary ca. 0.35 M Crawled parallel data ParaCrawl 5.1 ca. 34.37 M CommonCrawl ca. 2.40 M WikiMatrix ca. 6.22 M WikiTitles Monolingual crawled news dataca. 1.38 M German ca. 327.69 M English ca. 233.50 M 2.1 Round 1: Training models for scoring crawled parallel data In the first round, we trained base transformer models (Vaswani et al., 2017) with Sentence- Piece subword segmentation (Kudo and Richard- son, 2018) with a joint vocabulary of 32K tokens on the high-quality parallel data. The joint vocabu- lary remains the same for all models. We applied a binomial sentence length model to remove from the parallel data sentence pairs with an unreasonable sentence length ratio. The model assumes that a pair of sentences of lengths K and L is produced by a series of K+L flips of a biased coin. The bias is based on the corpus-level ratio of tokens; for Ger- man and English, we determined that there are on average 1.0723 English tokens per German token, so the Null Hypothesis assumes that an English word is generated with a probability of 51.75%, and a German word with a probability of 48.25%. For each sentence pair, we determine the p-value of the Null Hypothesis; if it is less than 0.5%, the sentence pair is discarded. This sentence filtering filtered out less than 2% of the EuroParl data, ca 3.4% of the NewsCommentary data, and 11% of the Rapid corpus. The numbers given in Table 1 are after filtering. 2.2 Selecting crawled parallel data We used these models to compute the length- normalized cross-entropy for each sentence pair in the available crawled parallel data in both trans- lation directions, and combined these two en- tropies into the dual cross-entropy score (Junczys- Dowmunt, 2018a). To bias data selection to- wards the news domain, we also computed length- normalized cross-entropy for each sentence with a 5-gram language model1 trained on the respec- tive monolingual news …

Content truncated to 500 words