OpenNMT: Neural Machine Translation Toolkit

OpenNMT: Neural Machine Translation Toolkit Guillaume Klein SYSTRAN Yoon Kim Harvard University Yuntian Deng Harvard University Vincent Nguyen Ubiqus Jean Senellart SYSTRAN Alexander M. Rush Harvard University Abstract OpenNMT is an open-source toolkit for neural machine translation (NMT). The system prior- itizes efficiency, modularity, and extensibility with the goal of supporting NMT research into model architectures, feature representations, and source modalities, while maintaining compet- itive performance and reasonable training requirements. The toolkit consists of modeling and translation support, as well as detailed pedagogical documentation about the underlying tech- niques. OpenNMT has been used in several production MT systems, modified for numerous research papers, and is implemented across several deep learning frameworks. 1 Introduction Neural machine translation (NMT) is a new methodology for machine translation that has led to remarkable improvements, particularly in terms of human evaluation, compared to rule-based and statistical machine translation (SMT) systems (Wu et al., 2016; Crego et al., 2016). Orig- inally developed using pure sequence-to-sequence models (Sutskever et al., 2014; Cho et al., 2014) and improved upon using attention-based variants (Bahdanau et al., 2014; Luong et al., 2015a), NMT has now become a widely-applied technique for machine translation, as well as an effective approach for other related NLP tasks such as dialogue, parsing, and summarization. As NMT approaches are standardized, it becomes more important for the machine trans- lation and NLP community to develop open implementations for researchers to benchmark against, learn from, and extend upon. Just as the SMT community benefited greatly from toolk- its like Moses (Koehn et al., 2007) for phrase-based SMT and CDec (Dyer et al., 2010) for syntax-based SMT, NMT toolkits can provide a foundation to build upon. A toolkit should aim to provide a shared framework for developing and comparing open-source systems, while at the same time being efficient and accurate enough to be used in production contexts. With these goals in mind, in this work we present an open-source toolkit for developing neural machine translation systems, known as OpenNMT (http://opennmt.net). Since its launch in December 2016, OpenNMT has become a collection of implementations targeting both academia and industry. The system is designed to be simple to use and easy to extend, while maintaining efficiency and state-of-the-art accuracy. In addition to providing code for the core translation tasks, OpenNMT was designed with two aims: (a) prioritize training and test efficiency, (b) maintain model modularity and readability hence research extensibility. During this time, many other stellar open-source NMT implementations have also been Proceedings of AMTA 2018, vol. 1: MT Research Track Boston, March 17 - 21, 2018 | Page 177 (a) (b) Figure 1: (a). Schematic view of neural machine translation. The red source words are first mapped to word vectors and then fed into a recurrent neural network (RNN). Upon seeing the ⟨eos⟩ symbol, the final time step initializes a target blue RNN. At each target time step, attention is applied over the source RNN and combined with the current hidden state to produce a prediction p(wt|w1:t−1, x) of the …

Content truncated to 500 words