Wild Devs’ at SemEval-2017 Task 2: Using Neural Networks to Discover Word Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 267–270, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics Wild Devs’ at SemEval-2017 Task 2: Using Neural Networks to Discover Word Similarity Răzvan-Gabriel Rotari, Ionuț Hulub, Ștefan Oprea, Mihaela Plămadă-Onofrei, Alina Beatrice Lorenţ, Raluca Preisler, Adrian Iftene, Diana Trandabăț University Alexandru Ioan Cuza of Iași, Romania {razvan.rotari, ionut.hulub, stefan.oprea, mihaela.onofrei, alina.lorent, raluca.preisler, adiftene, dtrandabat}@info.uaic.ro Abstract This paper presents Wild Devs’ participa- tion in the SemEval-2017 Task 2 “Multi- lingual and Cross-lingual Semantic Word Similarity”, which tries to automatically measure the semantic similarity between two words. The system was build using neural networks, having as input a collec- tion of word pairs, whereas the output consists of a list of scores, from 0 to 4, corresponding to the degree of similarity between the word pairs. 1 Introduction The Wild Dev’s team participated this year in SemEval 2017 Task 2, subtask 1, in the evalua- tion for the English language. The system is based on a neural network, trained on an enriched corpus of word pairs. The paper is structured in 4 sections: this sec- tion discusses existing approaches to similarity us- ing word embedding, before presenting the archi- tecture of our system in Section 2. The next sec- tion briefly analyses the results, while Section 4 drafts some conclusions and further work. In natural language processing, one of the most important challenges is to understand the meaning of words. The organizers of Task 2 (Task2, 2017) state that this task “provides a reliable benchmark for the development, evaluation and analysis” of:  Word embeddings, monolingual word embeddings, as well as bilingual and multilin- gual word embeddings which have a unified semantic space for the languages;  Similarity measures that use lexical resources;  Supervised systems that combine multiple measures. Our initial option was word embedding. The most prominent word embedding software tools are: 1. Word2Vec (Mikolov et al., 20013) is an algo- rithm with the explicit goal of producing word embeddings that encode general semantic rela- tions (Collobert et al., 2011). 2. GloVe (Pennington et al., 2014) has a similar aim as Word2Vec. Its authors present GloVe main- ly as an unsupervised learning algorithm, also of- fering an implementation. 3. Deeplearning4j is an open source deep learn- ing library for Java which implements both Word2Vec and GloVe, among other algorithms (Deeplearning4j, 2017). 4. Principal Component Analysis (Jolliffe, 2002) and T-Distributed Stochastic Neighbor Em- bedding (van der Maaten 2008) are two algo- rithms that reduce the dimensionality of already generated word embedding vectors. After initial tests using the data provided by the task organizers, we realized that it would yield better results to aggregate multiple techniques, and thus resolved to use a supervised system which combines multiple techniques. Recent stud- ies show that neural-network-inspired word em- bedding models exceed the traditional count- based distributional models on word similarity. 267 The supervised system we propose is a neural network trained on a list of gold standard word pairs. …

Content truncated to 500 words