DTSim at SemEval-2016 Task 1: Semantic Similarity Model Including Multi-Level Alignment and Vector-Based Compositional Semantics

Proceedings of SemEval-2016, pages 640–644, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics DTSim at SemEval-2016 Task 1: Semantic Similarity Model Including Multi-Level Alignment and Vector-Based Compositional Semantics Rajendra Banjade, Nabin Maharjan, Dipesh Gautam, Vasile Rus Department of Computer Science / Institute for Intelligent Systems The University of Memphis Memphis, TN, USA {rbanjade, nmharjan, dgautam, vrus}@memphis.edu Abstract In this paper we describe our system (DT- Sim) submitted at SemEval-2016 Task 1: Se- mantic Textual Similarity (STS Core). We developed Support Vector Regression model with various features including the similarity scores calculated using alignment based meth- ods and semantic composition based methods. The correlations between our system output and the human ratings were above 0.8 in three datasets. 1 Introduction The task of measuring the Semantic Textual Sim- ilarity (STS) is to quantify the degree of semantic similarity between the given pair of texts. For exam- ple, the similarity score of 0 means that the texts are not similar at all and 5 means that they have same meaning (Agirre et al., 2015; Banjade et al., 2015). In this paper, we describe our system DTSim and the submitted three different runs in this year’s SemEval shared task on Semantic Textual Similarity English track (STS Core; Agirre et al. (2016)). We applied Support Vector Regression (SVR) with various fea- tures in order to predict the similarity score for the given sentence pairs. The features of the model included semantic similarity scores calculated us- ing individual methods (described in Section 3) and other general features. The pipeline of components in DTSim is shown in Figure 1. 2 Preprocessing Hyphens were replaced with whitespaces if they were not composite verbs (e.g. video-gamed). The Figure 1: Pipeline of components in DTSim system. composite verbs were detected based on the POS tag assigned by the POS tagger. Also, the words starting with co-, pre-, meta-, multi-, re-, pro-, al-, anti-, ex-, and non- were left intact. Then, the hyphen-removed texts were tokenized, lemmatized, POS-tagged and annotated with Named Entity tags using Stanford CoreNLP Toolkit (Manning et al., 2014). We also marked each word as whether it was a stop word. We also created chunks using our own Conditional Ran- dom Fields (CRF) based chunking tool (Maharjan et al., 2016) which outperforms OpenNLP chunker when evaluated with human annotated chunks pro- vided in interpretable similarity shared task in 2015. We normalized texts using mapping data. For exam- ple, pct and % were changed to percent. 3 Feature Extraction We used various features in our regression models including semantic similarity scores generated using individual methods. Before describing those indi- vidual methods, we present word similarity methods which were used for sentence similarity calculation. 3.1 Word-to-Word Similarity We used vector based word representation models, PPDB 2.0 database (Pavlick et al., 2015), and Word- Net (Miller, 1995) in order to measure the similarity 640 between words as given below. sim(w1, w2, m) =            1, if w1 …

Content truncated to 500 words