yiGou: A Semantic Text Similarity Computing System Based on SVM

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 80–84, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics yiGou: A Semantic Text Similarity Computing System Based on SVM Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang School of Computer Science and Technology Harbin Institute of Technology, China {yliu,cjsun,linl,wangxl}@insun.hit.edu.cn Abstract This paper describes the yiGou system we de- veloped to compute the semantic similarity of two English sentences, which we submitted to the SemEval 2015 Task 2 (English subtask). The system uses a support vector machine model with literal similarity, shallow syntactic similarity, WordNet-based similarity and la- tent semantic similarity to predict the seman- tic similarity score of two short texts. In our experiments, WordNet-based and LSA-based features performed better than other features. Out of the 73 submitted runs, our two runs ranked 38th and 42th, with mean Pearson corre- lation 0.7114 and 0.6964 respectively. 1 Introduction Semantic Text Similarity (STS) plays an important role in many Natural language processing tasks, such as Question Answering (Narayanan and Harabagiu, 2004), Machine Translation (Beale et al., 1995), Automatic Summarization (Wang et al., 2008) and Word Sense Disambiguation (Navigli and Velardi, 2005). Since STS is an essential chal- lenge in NLP, that has attracted a significant amount of attention by the research community. SemEval has held tasks about STS for four years in a row, from which we can see the importance and difficulty of this challenge. Particularly, SemEval focuses on semantic similarity of short texts as a lot of researches about long texts have been done in past years and the demand of finding new meth- ods to measure short texts similarity has become stronger in many new applications. In this paper, we proposed a SVM-based solu- tion to compute the semantic similarity between two sentences which is the goal of SemEval 2015 Task 2. Knowledge-based and corpus-based fea- tures were involved in our solution. We used the combination of the word similarity to estimate sen- tence similarity. And the training data of SemEval 2012 (Agirre et al., 2012) was used to train our model. In our experiments, WordNet-based and LSA-based features performed better than other features. Out of the 73 submitted runs, our two runs ranked 38th and 42th, with mean Pearson cor- relation 0.7114 and 0.6964 respectively. The eval- uation results showed that our solution has good generalization ability on the test dataset of SemEval 2015 which is very different from our training set in terms of the sources of the sentences. Some of the relatively new technologies such as Word2Vec (Mikolov et al., 2013) and Sen- tence2Vec (Le and Mikolov, 2014) are potential methods to represent sentences and will be includ- ed in our further works. 2 Data and Metrics In SemEval 2015, the trial dataset comprises the 2012, 2013 and 2014 datasets, which can be used to develop and train models. Because of the limita- tion of the time, we only used the training data of SemEval 2012 as our training set. The …

Content truncated to 500 words