Neural Monkey: The Current State and Beyond

Neural Monkey: The Current State and Beyond Jindˇrich Helcl Jindˇrich Libovick´y Tom Kocmi Tom´aˇs Musil Ondˇrej C´ıfka Duˇsan Variˇs Ondˇrej Bojar [surname]@ufal.mff.cuni.cz Institute of Formal and Applied Linguistics Faculty of Mathematics and Physics, Charles University Malostransk´e n´amˇest´ı 25, Prague, 118 00, Czech Republic Abstract Neural Monkey is an open-source toolkit for sequence-to-sequence learning. The focus of this paper is to present the current state of the toolkit to the intended audience, which includes students and researchers, both active in the deep learning community and newcomers. For each of these target groups, we describe the most relevant features of the toolkit, including the simple configuration scheme, methods of model inspection that promote useful intuitions, or a modular design for easy prototyping. We summarize relevant contributions to the research community which were made using this toolkit and discuss the characteristics of our toolkit with respect to other existing systems. We conclude with a set of proposals for future development. 1 Introduction Neural Monkey (Helcl and Libovick´y, 2017) is an open-source tool for sequential learning de- veloped and maintained at Charles University. It is used mainly for experiments with neural machine translation (NMT), but also for other natural language processing (NLP) tasks which use deep learning, such as image captioning, multimodal translation, optical character recogni- tion, text summarization or sentiment analysis. In this paper, we present the state of the software at the beginning of 2018 and share the plans for future development. Neural Monkey is best suited for research and education. Our ambition is not developing a production-ready software. We also do not focus on technical tricks to achieve the best per- formance. Instead, we provide an extensible library of implemented features with references to the papers where they were proposed. We try to align our aims with the needs of three possibly overlapping groups of users. First, we want to help students understand how various models work and build intuition for their training. Second, we want to provide a simple tool for newcomers to the field of deep learning to help them start experimenting with these models. Third, we design the toolkit in a highly modular fashion with replaceable components to suit researchers who want to try out their ideas without having to implement everything from scratch. Proceedings of AMTA 2018, vol. 1: MT Research Track Boston, March 17 - 21, 2018 | Page 168 Neural Monkey is implemented in Python 3.5 using the TensorFlow library (Abadi et al., 2016). We share the source code on GitHub,1 where we also report and solve issues, provide support for users and discuss the development, so everyone can contribute with their ideas. This paper is organized as follows. Section 2 presents the goals of the development of the toolkit with respect to the needs of the considered user groups. Section 3 gives an overview of the toolkit features. In Section 4, we summarize the research contributions that have been made with Neural Monkey. We discuss the differences from other toolkits in Section 5. We …

Content truncated to 500 words