MultiVec: a Multilingual and Multilevel Representation Learning Toolkit for NLP

MultiVec: a Multilingual and Multilevel Representation Learning Toolkit for NLP Alexandre B´erard†‡, Christophe Servan‡, Olivier Pietquin†∗ and Laurent Besacier‡∗ †Univ. Lille, CNRS, Centrale Lille, Inria UMR 9189 - CRIStAL, F-59000 Lille, France alexandre.berard@ed.univ-lille1.fr olivier.pietquin@univ-lille1.fr ‡LIG, Univ. Grenoble Alpes Campus Saint-Martin d’H`eres, Grenoble, France christophe.servan@imag.fr laurent.besacier@imag.fr ∗Institut Universitaire de France Abstract We present MultiVec, a new toolkit for computing continuous representations for text at different granularity levels (word-level or sequences of words). MultiVec includes Mikolov et al. [2013b]’s word2vec features, Le and Mikolov [2014]’s paragraph vector (batch and online) and Luong et al. [2015]’s model for bilingual distributed representations. MultiVec also includes different distance measures between words and sequences of words. The toolkit is written in C++ and is aimed at being fast (in the same order of magnitude as word2vec), easy to use, and easy to extend. It has been evaluated on several NLP tasks: the analogical reasoning task, sentiment analysis, and crosslingual document classification. Keywords: Word embeddings, paragraph vector, bilingual word embeddings, crosslingual document classification 1. Introduction There has been a growing interest in distributed representa- tions for text, largely due to Mikolov et al. [2013a] who pro- pose simple models which can be trained on huge amounts of data. A number of contributions have extended this work to phrases [Mikolov et al., 2013b], text sequences [Le and Mikolov, 2014], bilingual distributed representations [Lu- ong et al., 2015] [Gouws et al., 2015], or bilingual repre- sentations for text sequences [Pham et al., 2015]. Although most of these techniques have official or non-official implementations (word2vec, bivec, gensim [ ˇReh˚uˇrek and Sojka, 2010], etc.), there has been no con- certed effort to regroup all of these techniques in a single toolkit. Contribution This paper presents MultiVec, a toolkit which enables the generation and manipulation of multi- lingual vector representations at several granularity levels (from word to any sequence of words). MultiVec com- bines several techniques of the literature: it includes most of word2vec’s features [Mikolov et al., 2013a] for learning distributed word representations (also known as word em- beddings); as well as an implementation of Le and Mikolov [2014]’s paragraph vector. In addition, MultiVec can com- pute bilingual representations on a parallel corpus using Lu- ong et al. [2015]’s bivec model. The code (provided on GitHub) is written in C++, and is fast, easy to use and read- able. The models can also be used from Python code thanks to a Python wrapper. We provide the results and code for a number of compar- isons and benchmarks that test the usability of this toolkit against other toolkits in the literature. The MultiVec toolkit has two main components: the first component enables the generation of new models, while the second component uses those models to compute distances between words or sequences. It also includes a series of benchmarks that makes possible the evaluation of trained models on different NLP tasks. Outline The rest of this paper goes simply as follows: we first describe the models that can be trained using Multi- Vec. …

Content truncated to 500 words