Sbdlrhmn: A Rule-based Human Interpretation System for Semantic Textual Similarity Task

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 536–542, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Sbdlrhmn: A Rule-based Human Interpretation System for Semantic Textual Similarity Task Samir AbdelRahman sbdlrhmn@illinois.edu, s.abdelrahman@fci-cu.edu.eg Catherine Blake clblake@illinois.edu The Graduate School of Library and Information Science University Of Illinois at Urbana-Champaign Abstract In this paper, we describe the system architec- ture used in the Semantic Textual Similarity (STS) task 6 pilot challenge. The goal of this challenge is to accurately identify five levels of semantic similarity between two sentences: equivalent, mostly equivalent, roughly equiva- lent, not equivalent but sharing the same topic and no equivalence. Our participations were two systems. The first system (rule-based) combines both semantic and syntax features to arrive at the overall similarity. The proposed rules enable the system to adequately handle domain knowledge gaps that are inherent when working with knowledge resources. As such one of its main goals, the system sug- gests a set of domain-free rules to help the human annotator in scoring semantic equiva- lence of two sentences. The second system is our baseline in which we use the Cosine Simi- larity between the words in each sentence pair. 1 Introduction Accurately establishing sentence semantic similari- ty would provide one of the key ingredients for solutions to many text-related applications, such as automatic grading systems (Mohler and Mihalcea, 2009), paraphrasing (Fernando and Stevenson, 2008), text entailment (Corley et al., 2005) and summarization (Erkan and Radev, 2004). Current approaches for computing semantic similarity be- tween a pair of sentences focus on analyzing their shared words (Salton, 1989), structures (Hu et al. 2011;Mandreoli et al. 2002), semantics (Mihalcea et al. 2006; Le el al. 2006; Hatzivassiloglou, 1999) or any of their combinations (Liu et al. 2008; Foltz et al. 1998). The goal is to arrive at a score which increases proportionally with the relatedness be- tween the two sentences. Yet, they are not con- cerned with scoring the interpretations of such relatedness (Zhang et al. 2011; Jesus et al. 2011; Wenyin et al. 2010; Liu et al. 2008). Semantic Textual Similarity (STS), SEMEVAL- 12 Task 6 (Agirre et al. 2012), measures the degree of semantic equivalence between a pair of sentenc- es by comparing meaningful contents within a sen- tence. The assigned scores range from 0 to 5 for each sentence pair with the following interpreta- tions: (5) completely equivalent, (4) mostly equivalent pair with missing unimportant infor- mation, (3) roughly equivalent with missing im- portant information, (2) not equivalent, but sharing some details, (1) not equivalent but sharing the same topic and (0) not equivalent and on different topics. The goal of developing our rule-based sys- tem was to identify knowledge representations which have possibly all task human interpretations. Meanwhile, the system domain-free rules aim to help the human annotator in scoring semantic equivalence of sentence pair. The proposed rule-based solution exploits both sentence syntax and semantics. First, it uses Stan- ford parser (Klein and Manning, 2002) to expose the sentence structure, part-of-speech …

Content truncated to 500 words