LibN3L:A Lightweight Package for Neural NLP

LibN3L:A Lightweight Package for Neural NLP Meishan Zhang1, Jie Yang2, Zhiyang Teng2, Yue Zhang2 1.School of Computer Science and Technology, Heilongjiang University, Harbin, China 2. Singapore University of Technology and Design {meishan zhang, yue zhang}@sutd.edu.sg {jie yang, zhiyang teng}@mymail.sutd.edu.sg Abstract We present a light-weight machine learning tool for NLP research. The package supports operations on both discrete and dense vectors, facilitating implementation of linear models as well as neural models. It provides several basic layers which mainly aims for single-layer linear and non-linear transformations. By using these layers, we can conveniently implement linear models and simple neural models. Besides, this package also integrates several complex layers by composing those basic layers, such as RNN, Attention Pooling, LSTM and gated RNN. Those complex layers can be used to implement deep neural models directly. Keywords: neural network, sequence labeling, recurrent neural network 1. Introduction Deep learning methods have received increasing research attention in natural language processing (NLP), with neu- ral models being built for classification (Kalchbrenner et al., 2014), sequence labeling (Collobert et al., 2011), pars- ing (Socher et al., 2013; Dyer et al., 2015; Zhou et al., 2015; Weiss et al., 2015), machine translation (Cho et al., 2014), fine-grained sentiment analysis (Zhang et al., 2015) and other tasks. This surge of the interest gives rise to a demand of software libraries, which can facilitate research by allowing fast prototyping and modeling for experimen- tation. For traditional methods such as conditional random field- s (CRF) (Lafferty et al., 2001) and SVM (Vapnik, 1995), there has been various software toolkits, implemented in different programming languages, including Java, Python and C++. These toolkits offer a large degree of variety for building NLP models by using or adapting the ma- chine learning algorithms. For deep learning, a number of software tools have been developed, including theano 1 (Bergstra et al., 2010), caffe 2 (Jia et al., 2014), CNN3, torch4 etc. These tools are based on different programming languages and design concepts. On the other hand, most of these libraries are not designed specifically for NLP tasks. In addition, many existing libraries define a complex class hierarchy, making it difficult for some users to use or adapt the modules. We present another deep learning toolkit in C++, designed specifically for NLP applications. The main objective is to make it extremely light-weight, so as to minimize the effort in building a neural model. We take a layered ap- proach, offering high-level models for classification and sequence labeling, such as neural CRF (Do et al., 2010), recurrent neural networks (RNN) (Graves, 2012) and long- short-term memories (LSTM) (Hochreiter and Schmidhu- ber, 1997), which are frequently used in NLP. On the oth- 1https://github.com/Theano/Theano 2http://caffe.berkeleyvision.org/ 3https://github.com/clab/cnn 4http://torch.ch/ er hand, we minimize encapsulation, implementing neural structures strictly abiding by their formal definitions, so as to make it easy to work directly with neural layers and fa- cilitate extensions to existing network structures. Our design is centralized in the structure of a neural lay- er, which performs the standard feed-forward function and …

Content truncated to 500 words