The OpenNMT Neural Machine Translation Toolkit: 2020 Edition

The OpenNMT Neural Machine Translation Toolkit: 2020 Edition Guillaume Klein guillaume.klein@systrangroup.com Franc¸ois Hernandez fhernandez@ubiqus.com Vincent Nguyen vnguyen@ubiqus.com Jean Senellart jean.senellart@systrangroup.com Abstract OpenNMT is a multi-year open-source ecosystem for neural machine translation (NMT) and natural language generation (NLG). The toolkit consists of multiple projects to cover the com- plete machine learning workflow: from data preparation to inference acceleration. The systems prioritize efficiency, modularity, and extensibility with the goal of supporting research into model architectures, feature representations, and source modalities, while maintaining API sta- bility and competitive performance for production usages. OpenNMT has been used in several production MT systems and cited in more than 700 research papers. 1 Introduction Neural machine translation (NMT) is a recent approach for machine translation that has led to remarkable improvements, particularly in terms of human evaluation, compared to rule- based and statistical machine translation (SMT) systems (Wu et al., 2016). From the initial work on recurrent sequence-to-sequence models (Sutskever et al., 2014) to recent advances on self-attentional models (Vaswani et al., 2017), a significant amount of contributions explored various model architectures, hyper-parameter settings, and data preparation techniques. This exploration is often constrained by engineering issues related to computation efficiency, code design, and model deployment. In this context, we introduced OpenNMT 1 in late 2016, the first large audience open- source toolkit to design, train, and deploy neural machine translation models. The OpenNMT initiative consists of several projects to assist researchers and developers in their NMT journey, from data preparation to inference acceleration. It supports a wide range of model architectures and training procedures for neural machine translation as well as related tasks such as natural language generation and language modeling. The open source community around neural machine translation is very active and includes several other projects that have similar goals and capabilities such as Fairseq (Ott et al., 2019), Sockeye (Hieber et al., 2017), or Marian (Junczys-Dowmunt et al., 2018). In the ongoing de- velopment of OpenNMT, we aim to build upon the strengths of those systems, while providing unique features and technology support. In this paper, we briefly give an overview of the OpenNMT project and its history. We then present the key features that make OpenNMT particularly suited for research and production. Finally, we share some benchmarks for comparison and current work directions. 1https://opennmt.net Proceedings of the 14th Conference of the Association for Machine Translation in the Americas October 6 - 9, 2020, Volume 1: MT Research Track Page 102 2 Project overview OpenNMT is a collection of projects supporting easy adoption of neural machine translation and related tasks. The project has two actively maintained implementations: • OpenNMT-py: A user-friendly and multimodal implementation benefiting from PyTorch ease of use and versatility. • OpenNMT-tf: A modular and stable implementation powered by the TensorFlow 2 ecosystem. These implementations provide command line utilities and a Python library to configure, train, and run models. Each implementation has its own design and set of features, but both share the goals that started the OpenNMT initiative: ease of use, …

Content truncated to 500 words