Results of the WMT17 Neural MT Training Task

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 525–533 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Results of the WMT17 Neural MT Training Task Ondˇrej Bojar Jindˇrich Helcl Tom Kocmi Jindˇrich Libovick´y Tom´aˇs Musil Charles University, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Malostransk´e n´amˇest´ı 25, 118 00 Prague, Czech Republic <surname>@ufal.mff.cuni.cz Abstract This paper presents the results of the WMT17 Neural MT Training Task. The objective of this task is to explore the methods of training a fixed neural ar- chitecture, aiming primarily at the best translation quality and, as a secondary goal, shorter training time. Task par- ticipants were provided with a complete neural machine translation system, fixed training data and the configuration of the network. The translation was performed in the English-to-Czech direction and the task was divided into two subtasks of dif- ferent configurations—one scaled to fit on a 4GB and another on an 8GB GPU card. We received 3 submissions for the 4GB variant and 1 submission for the 8GB vari- ant; we provided also our run for each of the sizes and two baselines. We translated the test set with the trained models and evaluated the outputs using several auto- matic metrics. We also report results of the human evaluation of the submitted sys- tems. 1 Introduction Neural machine translation (NMT) has recently replaced the “classical statistical machine transla- tion” and became the dominant research paradigm. A large part of research on NMT is focused on ar- chitectural improvements of the neural networks or data preprocessing. However, in practice, the results of an NMT system depends not only on the architecture of the network, but also on the train- ing techniques used to obtain the parameters. The goal of NMT Training Task1 is to compare 1http://www.statmt.org/wmt17/ nmt-training-task/ the results of various training techniques applied to a fixed network architecture. We provided task participants with the model specification, training and validation data with a fixed way of data pre- processing. We also listed a few methods as an inspiration for the participants. These methods in- cluded (but were not limited to) the following: Curricula. The basic idea behind this technique (Bengio et al., 2009) is inspired by the fact that humans learn more easily when examples are pre- sented in an ordering from trivial to complex ones. Neural networks could potentially also benefit from such a strategy of increasing task difficulty. This technique includes modifications of the train- ing data in order to converge faster, or more ro- bustly, towards possibly better local optima. Data shuffling, reordering, or back-translation (Sen- nrich, Haddow, and Birch, 2016a), are all tech- niques that can have a positive impact on the train- ing. Optimization algorithms. There are many opti- mization algorithms that can be employed to train- ing an NMT model, such as Adadelta (Zeiler, 2012) or Adam (Kingma and Ba, 2014). Each method differs in the number of inner trainable parameters and the approach it …

Content truncated to 500 words