QLUT at SemEval-2017 Task 2: Word Similarity Based on Word Embedding and Knowledge Base

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 235–238, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics QLUT at SemEval-2017 Task 2: Word Similarity Based on Word Embedding and Knowledge Base Fanqing Meng1, Wenpeng Lu*1, Yuteng Zhang1,Ping Jian2,Shumin Shi2,Heyan Huang2 1School of Information, QiLu University of Technology, Jinan, Shandong, China 2School of Computer, Beijing Institute of Technology, Beijing, China mengfanqing678@163.com, lwp@qlu.edu.cn, zhangyuteng1029@163.com, pjian@bit.edu.cn, bjssm@bit.edu.cn, hhy63@bit.edu.cn Abstract This paper shows the details of our system submissions in the task 2 of SemEval 2017. We take part in the subtask 1 of this task, which is an English monolingual sub- task. This task is designed to evaluate the semantic word similarity of two linguistic items. The results of runs are assessed by standard Pearson and Spearman correla- tion, contrast with official gold standard set. The best performance of our runs is 0.781 (Final). The techniques of our runs mainly make use of the word embeddings and the knowledge-based method. The re- sults demonstrate that the combined meth- od is effective for the computation of word similarity, while the word embeddings and the knowledge-based technique, respec- tively, needs more deeply improvement in details. 1 Introduction Semantic word similarity aims at measuring the extent to which two words are similar (Camacho- Collados et al., 2017). Given two words, the runs in this competition should give a score which in- dicates the similarity between them, and it will be evaluated by the official gold standard set. This task doesn’t offer any annotated corpus and the organizers encourage systems to utilize unlabeled corpus. With the development of word embed- dings technique, more and more attentions are paid to it (Mikolov et al., 2013a; Mikolov et al., 2013b). We also adopt the word embeddings ________________________ *Corresponding author method in our runs. Besides the word embeddings method, another knowledge-based method is proposed by us, which is based on BabelNet (Navigli and Ponzetto, 2012). Integrating Wikipedia and WordNet, BabelNet is a multilingual encyclopedic and lexicographic knowledge base, which builds an enormous semantic network linking concepts and named entities with the aid of a large semantic relations. Based on the word embedding method and the knowledge-based method, a combined method is implemented, which achieves the best perfor- mance. 2 System Overview In the subtask 1 (English monolingual word simi- larity) of this task, we have submitted two system runs, both of which are unsupervised. We mainly utilize the word embeddings method and the com- bined method. The Figure 1 shows the framework of our sys- tem runs. In the top part of the figure, word1 and word2 are the input of our systems. Run1 utilizes the word embeddings method. Run2 utilizes the combined method, which is based on the word embeddings and knowledge-based method. 2.1 DataSet Test Set: In this task, we submit our runs on the English monolingual word similarity dataset, which includes 500 word pairs. These word pairs 235 may be concepts or named entities, which are tab- separated. Gold Standard …

Content truncated to 500 words