XNMT: The eXtensible Neural Machine Translation Toolkit

XNMT: The eXtensible Neural Machine Translation Toolkit Graham Neubig Carnegie Mellon University, Pittsburgh, USA Matthias Sperber Karlsruhe Institute of Technology, Karlsruhe, Germany Xinyi Wang, Matthieu Felix, Austin Matthews, Sarguna Padmanabhan, Ye Qi, Devendra Singh Sachan Carnegie Mellon University, Pittsburgh, USA Philip Arthur Nara Institute of Science and Technology, Nara, Japan Pierre Godard LIMSI/CNRS/Universit´e Paris-Saclay, Orsay, France John Hewitt University of Pennsylvania, Philadelphia, USA Rachid Riad ENS/CNRS/EHESS/INRIA, Paris, France Liming Wang University of Illinois, Urbana-Champaign, USA Abstract This paper describes XNMT, the eXtensible Neural Machine Translation toolkit. XNMT distin- guishes itself from other open-source NMT toolkits by its focus on modular code design, with the purpose of enabling fast iteration in research and replicable, reliable results. In this paper we describe the design of XNMT and its experiment configuration system, and demonstrate its util- ity on the tasks of machine translation, speech recognition, and multi-tasked machine transla- tion/parsing. XNMT is available open-source at https://github.com/neulab/xnmt. 1 Introduction Due to the effectiveness and relative ease of implementation, there is now a proliferation of toolkits for neural machine translation (Kalchbrenner and Blunsom, 2013; Sutskever et al., 2014; Bahdanau et al., 2015), as many as 51 according to the tally by nmt-list.1 The com- mon requirements for such toolkits are speed, memory efficiency, and translation accuracy, which are essential for the use of such systems in practical translation settings. Many open source toolkits do an excellent job at this to the point where they can be used in production systems (e.g. OpenNMT2 is used by Systran (Crego et al., 2016)). This paper describes XNMT, the eXtensible Neural Machine Translation toolkit, a toolkit that optimizes not for efficiency, but instead for ease of use in practical research settings. In other words, instead of only optimizing time for training or inference, XNMT aims to reduce the time it takes for a researcher to turn their idea into a practical experimental setting, test with a large number of parameters, and produce valid and trustable research results. Of course, this necessitates a certain level of training efficiency and accuracy, but XNMT also takes into account a number of considerations, such as those below: • XNMT places a heavy focus on modular code design, making it easy to swap in and out different parts of the model. Ideally, implementing research prototypes with XNMT in- volves only few changes to existing code. 1https://github.com/jonsafari/nmt-list 2http://opennmt.net Proceedings of AMTA 2018, vol. 1: MT Research Track Boston, March 17 - 21, 2018 | Page 185 • XNMT is implemented in Python, the de facto standard in the research community. • XNMT uses DyNet (Neubig et al., 2017) as its deep learning framework. DyNet uses dy- namic computation graphs, which makes it possible to write code in a very natural way, and benefit from additional flexibility to implement complex networks with dynamic structure, as are often beneficial in natural language processing. Further benefits include transparent handling of batching operations, or even removing explicit batch handling and relying on autobatching for speed-up instead. • XNMT of …

Content truncated to 500 words