NeRoSim: A System for Measuring and Interpreting Semantic Textual Similarity

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 164–171, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics NeRoSim: A System for Measuring and Interpreting Semantic Textual Similarity Rajendra Banjade∗, Nobal B. Niraula∗, Nabin Maharjan∗, Vasile Rus, Dan Stefanescu†, Mihai Lintean†, Dipesh Gautam Department of Computer Science The University of Memphis Memphis, TN {rbanjade,nbnraula,nmharjan,vrus,dstfnscu,mclinten,dgautam}@memphis.edu Abstract We present in this paper our system developed for SemEval 2015 Shared Task 2 (2a - En- glish Semantic Textual Similarity, STS, and 2c - Interpretable Similarity) and the results of the submitted runs. For the English STS subtask, we used regression models combin- ing a wide array of features including semantic similarity scores obtained from various meth- ods. One of our runs achieved weighted mean correlation score of 0.784 for sentence similar- ity subtask (i.e., English STS) and was ranked tenth among 74 runs submitted by 29 teams. For the interpretable similarity pilot task, we employed a rule-based approach blended with chunk alignment labeling and scoring based on semantic similarity features. Our system for interpretable text similarity was among the top three best performing systems. 1 Introduction Semantic Textual Similarity (STS) is the task of measuring the degree of semantic equivalence for a given pair of texts. The importance of semantic similarity in Natural Language Processing is high- lighted by the diversity of datasets and shared task evaluation campaigns over the last decade (Dolan et al., 2004; Agirre et al., 2012; Agirre et al., 2013; Agirre et al., 2014; Rus et al., 2014) and by many uses such as in text summarization (Aliguliyev, 2009) and student answer assessment (Rus and Lin- tean, 2012; Niraula et al., 2013). ∗* These authors contributed equally to this work ††Work done while at University of Memphis This year’s SemEval shared task on semantic textual similarity focused on English STS, Span- ish STS, and Interpretable Similarity (Agirre et al., 2015). We participated in the English STS and In- terpretable Similarity subtasks. We describe in this paper our systems participated in these two subtasks. The English STS subtask was about assigning a similarity score between 0 and 5 to pairs of sen- tences; a score of 0 meaning the sentences are un- related and 5 indicating they are equivalent. Our three runs for this subtask combined a wide array of features including similarity scores calculated us- ing knowledge based and corpus based methods in a regression model (cf. Section 2). One of our sys- tems achieved mean correlation score of 0.784 with human judgment on the test data. Although STS systems measure the degree of se- mantic equivalence in terms of a score which is use- ful in many tasks, they stop short of explaining why the texts are similar, related, or unrelated. They do not indicate what kind of semantic relations ex- ist among the constituents (words or chunks) of the target texts. Finding explicit relations between con- stituents in the paired texts would enable a mean- ingful interpretation of the similarity scores. To …

Content truncated to 500 words