Samsung and University of Edinburgh’s System for the IWSLT 2018 Low Resource MT Task

Samsung and University of Edinburgh’s System for the IWSLT 2018 Low Resource MT Task Philip Williams2, Marcin Chochowski1, Pawel Przybysz1, Rico Sennrich2, Barry Haddow2, Alexandra Birch2 1Samsung R&D Institute, Poland 2School of Informatics, University of Edinburgh {m.chochowski,p.przybysz}@samsung.com {pwillia4,bhaddow}@inf.ed.ac.uk, {rico.sennrich,a.birch}@ed.ac.uk Abstract This paper describes the joint submission to the IWSLT 2018 Low Resource MT task by Samsung R&D Institute, Poland, and the University of Edinburgh. We focused on supplement- ing the very limited in-domain Basque-English training data with out-of-domain data, with synthetic data, and with data for other language pairs. We also experimented with a vari- ety of model architectures and features, which included the development of extensions to the Nematus toolkit. Our sub- mission was ultimately produced by a system combination in which we reranked translations from our strongest individual system using multiple weaker systems. 1. Introduction This paper describes the joint submission to the IWSLT 2018 Low Resource MT task by Samsung R&D Institute, Poland, and the University of Edinburgh. We built several multilin- gual systems using the Tensor2Tensor1 and Nematus2 toolk- its, ultimately choosing to use a system combination in which we reranked translations from our strongest individual sys- tem using multiple weaker systems. As there was so little in-domain Basque-English data available, we experimented with the use of out-of-domain data, with the addition of synthetic data via back-translation, and with the incorporation of data for other language pairs. To support multilingual translation, we followed the single- model approach of [1] and simply prepended each source sentence with a token specifying the target language. We experimented with a variety of model architectures and features. This involved the development of several ex- tensions to the Nematus toolkit, including support for multi- GPU training, label smoothing, and mixtures of softmaxes. We have contributed our code to the public Nematus reposi- tory. 1https://github.com/tensorflow/tensor2tensor: 1.6.3 2https://github.com/EdinburghNLP/nematus 2. Training Data In brief, we used all of the provided in-domain parallel train- ing data along with parallel data from OpenSubtitles and the Open Data Euskadil Repository. We also produced synthetic data by back-translating from English into Basque. Table 1 lists the individual parallel corpora that made up our training data. Note that not all of our systems used all of the data. We will indicate differences when describing the individual systems. 2.1. In-Domain Data We used all of the available in-domain data, which we filtered in order to remove the TED talks covered by the devset. The task organisers had already removed the devset talks from the Basque-English training corpus, but the talks were present in the training data for all other language pairs. Since we were evaluating on the devset during system development, we fil- tered the in-domain data to avoid being misled by artificially strong results. In preliminary multilingual Nematus systems that used only the in-domain data, this filtering had a signifi- cant impact, reducing the BLEU score from 16.25 to 9.96. For the Basque-Spanish and Spanish-English pairs, we used the excised training data to create supplementary de- vsets. Since the Basque-English …

Content truncated to 500 words