NNBlocks: A Deep Learning Framework for Computational Linguistics Neural Network Models

NNBlocks: A Deep Learning Framework for Computational Linguistics Neural Network Models *Frederico Tommasi Caroli, *Jo˜ao Carlos Pereira da Silva, **Andr´e Freitas, **Siegfried Handschuh *Universidade Federal do Rio de Janeiro, **Universit¨at Passau *Rio de Janeiro, **Passau ftcaroli@gmail.com, jcps@dcc.ufrj.br, andre.freitas@uni-passau.de, siegfried.handschuh@uni-passau.de Abstract Lately, with the success of Deep Learning techniques in some computational linguistics tasks, many researchers want to explore new models for their linguistics applications. These models tend to be very different from what standard Neural Networks look like, limiting the possibility to use standard Neural Networks frameworks. This work presents NNBlocks, a new framework written in Python to build and train Neural Networks that are not constrained by a specific kind of architecture, making it possible to use it in computational linguistics. Keywords: Deep Learning, Artificial Neural Network, Computational Linguistics 1. Introduction Neural Networks (NN) are a kind of statistical learning methods that has been gaining a lot of attention with the appearance of a variety of techniques that make possible the so-called Deep Learning (DL). DL happens when a statistical model, normally a NN, learns layers of high- level abstractions in the data. This idea became popular in 2006(Hinton et al., 2006), but just recently this kind of learning has being applied successfully in computational linguistics. This is partially driven by the fact that linguis- tics features are highly complex and tend to not fit in tra- ditional NNs architectures. The most notable example is Multilayer Perceptron, that cannot handle variable sized in- puts. With the creation of new kinds of NNs architectures, re- searchers can fit their linguistics tasks and experiment with them, but this comes with a great load of implementing the model and training it from scratch, since standard NNs frameworks are not always ready to deal with architectures that are very different from the standard ones applied for other tasks. NNBlocks is a Deep Learning framework written in Python that aims to solve this problem, being very expandable and easy to use, while providing state-of-the-art training tech- niques without a single modification to the model. The easy training of NNs is achieved by using Theano(Bergstra et al., 2010), a math expression compiler and automatic symbolic derivation framework, as workhorse of NNBlocks. This also gives NNBlocks the ability to run on GPU without any modifications to the code. NNBlocks makes it possible to implement and test NNs that are very different from stan- dard ones. The first section presents some related work, the second section discusses NNs architectures used for computational linguistics tasks, the third section presents the framework and the fourth concludes discussing how NNBlocks can achieve the previously discussed architectures. 2. Related Work Some Deep Learning frameworks with variable degrees of flexibility exists. In this section we will review some of them and discuss what points could be improved. The first framework we will talk about is Lasagne.1 This framework, like others, has an approach similar to NNBlocks, in which processing blocks are connected with each other to create a complete NN. This framework …

Content truncated to 500 words