UFAL Submissions to the IWSLT 2016 MT Track

UFAL Submissions to the IWSLT 2016 MT Track Ondˇrej Bojar, Ondˇrej C´ıfka, Jindˇrich Helcl, Tom Kocmi, and Roman Sudarikov Charles University, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics surname@ufal.mff.cuni.cz Abstract We present our submissions to the IWSLT 2016 machine translation task, as our first attempt to translate subtitles and one of our early experiments with neural machine translation (NMT). We focus primarily on English→Czech translation direction but perform also basic adaptation experiments for NMT with German and also the reverse direction. Three MT systems are tested: (1) our Chimera, a tight combination of phrase-based MT and deep linguistic processing, (2) Neural Monkey, our implementation of a NMT system in Tensor- Flow and (3) Nematus, an established NMT system. 1. Introduction In the machine translation task (MT task) at IWSLT 2016, the goal is to translate a given set of sentences using (a sub- set of) permitted training data and any MT system. IWSLT has covered a range of languages and recently started to in- clude Czech in one of the tested domains, namely the “Talk” domain in 2016. We take this opportunity to evaluate our system Chimera [1] on subtitles, and to compare it with two neural MT sys- tems: Neural Monkey [2] and the well-known Nematus [3]. The paper is structured as follows. In Section 2, we de- scribe our selection of the training data and note that unfortu- nately, the permitted resources for IWSLT this year included an overlap with the test sets. The subsequent sections are de- voted to the three system setups targetting Czech: Chimera in Section 3, Neural Monkey in Section 4 and Nematus in Sec- tion 5. The results are presented and discussed in Section 6. Since NMT is computationally expensive and requires spe- cial hardware (large-memory GPUs), we also used Amazon EC2 resources, see Section 7 for some technical details. 2. Training and Test Data The organizers of the MT track provided participants with cleaned development and test sets, and also with a relatively small in-domain training set of sentences. Additionally, a long list of permitted resources was given on the web page. To make a better use of the large set of permitted data, we selected sentences similar to the official in-domain training set, as described in Section 2.1. We spotted an overlap between CzEng 1.6, one of the permitted corpora, and the official development and test sets. In order to avoid overfitting (tuning phrase-based MT on part of its training data is likely to arrive at too strong preference for long phrases), we created our own version of the devel- opment set, as detailed in Section 2.2. We leave up to the organizers of the shared task to se- lect the best way to handle the overlap with the official test set. As described below, the overlap is not always at the level of sentences, so it is rather difficult to identify which sen- tences from the test set should be disregarded (and if a rea- sonably …

Content truncated to 500 words