Jmp8 at SemEval-2017 Task 2: A simple and general distributional approach to estimate word similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 230–234, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics Jmp8 at SemEval-2017 Task 2: A simple and general distributional approach to estimate word similarity Josué Melka LIASD - Université Paris 8 jmelka@ai.univ-paris8.fr Gilles Bernard LIASD - Université Paris 8 gb@ai.univ-paris8.fr Abstract We have built a simple corpus-based sy- stem to estimate words similarity in mul- tiple languages with a count-based appro- ach. After training on Wikipedia corpora, our system was evaluated on the multi- lingual subtask of SemEval-2017 Task 2 and achieved a good level of performance, despite its great simplicity. Our results tend to demonstrate the power of the dis- tributional approach in semantic similarity tasks, even without knowledge of the un- derlying language. We also show that di- mensionality reduction has a considerable impact on the results. 1 Introduction Despite the crucial importance of semantic simila- rity in NLP, the vast majority of experiments have been conducted on the English language, which raises the question whether the developed approa- ches can be generalized. SemEval-2017 Task 2 provides us with a fra- mework for evaluating semantic representations in multiple languages and compare them. We focus here on the multilingual subtask, which consists of five monolingual word similarity datasets. Our submission is based on the well known sta- tistical approach which uses bag-of-contexts re- presentation of words in a vector space model. We run two versions of our system, the first one using a direct sparse representation and the se- cond one with compressed dense representation (detailed below). This second version was evalua- ted after the official evaluation deadline, and pro- duced superior results as will appear below. We briefly describe the multilingual subtask in section 2. Next, in section 3, we detail our sy- stem and its parameters. The results are presented and analyzed in section 4, and then we conclude in section 5. 2 Task description Camacho-Collados et al. (2017) describes the task as follows: Given a pair of words, the task is to automatically measure their seman- tic similarity. All pairs in our datasets are scored according to a [0-4] simila- rity scale, where 4 denotes that the two words are synonymous and 0 indicates that they are completely dissimilar. Multilingual word similarity This subtask provides five monolingual word similarity datasets in English, German, Italian, Spanish and Farsi. The sub- task is intended to test not only mono- lingual approaches but also multilingual and language-independent techniques. The individual score of the systems is defined by the authors as the harmonic mean of Pearson and Spearman correlations on the corresponding dataset. However, as our analysis lead us to take into account the separate behavior of both measu- res, we did not focus here on the final score. 3 Our system Our system is corpus-based only, and uses a few well known ideas from the distributional approach in word semantic similarity. 3.1 The training corpus We have used the Wikipedia corpus taken from https://sites.google.com/site/rmyeid/ …

Content truncated to 500 words