IBM_EG-CORE: Comparing multiple Lexical and NE matching features in measuring Semantic Textual similarity

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 187–193, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics IBM_EG-CORE: Comparing multiple Lexical and NE matching features in measuring Semantic Textual similarity Sara Noeman IBM Cairo Technology and Development Center Giza, Egypt P.O. Box 166 Al-Ahram noemans@eg.ibm.com Abstract We present in this paper the systems we participated with in the Semantic Textual Similarity task at SEM 2013. The Semantic Textual Similarity Core task (STS) computes the degree of semantic equivalence between two sentences where the participant systems will be compared to the manual scores, which range from 5 (semantic equivalence) to 0 (no relation). We combined multiple text similarity measures of varying complexity. The experiments illustrate the different effect of four feature types including direct lexical matching, idf-weighted lexical matching, modified BLEU N-gram matching and named entities matching. Our team submitted three runs during the task evaluation period and they ranked number 11, 15 and 19 among the 90 participating systems according to the official Mean Pearson correlation metric for the task. We also report an unofficial run with mean Pearson correlation of 0.59221 on STS2013 test dataset, ranking as the 3rd best system among the 90 participating systems. 1 Introduction The Semantic Textual Similarity (STS) task at SEM 2013 is to measure the degree of semantic equivalence between pairs of sentences as a graded notion of similarity. Text Similarity is very important to many Natural Language Processing applications, like extractive summarization (Salton et al., 1997), methods for automatic evaluation of machine translation (Papineni et al., 2002), as well as text summarization (Lin and Hovy, 2003). In Text Coherence Detection (Lapata and Barzilay, 2005), sentences are linked together by similar or related words. For Word Sense Disambiguation, researchers (Banerjee and Pedersen, 2003; Guo and Diab, 2012a) introduced a sense similarity measure using the sentence similarity of the sense definitions. In this paper we illustrate the different effect of four feature types including direct lexical matching, idf-weighted lexical matching, modified BLEU N-gram matching and named entities matching. The rest of this paper will proceed as follows, Section 2 describes the four text similarity features used. Section 3 illustrates the system description, data resources as well as Feature combination. Experiments and Results are illustrated in section 4. then we report our conclusion and future work. 2 Text Similarity Features Our system measures the semantic textual similarity between two sentences through a number of matching features which should cover four main dimensions: i) Lexical Matching ii) IDF-weighted Lexical Matching iii) Contextual sequence Matching (Modified BLEU Score), and iv) Named Entities Matching. First we introduce the alignment technique used. For a sentence pair {s1, s2} matching is done in each direction separately to detect the sub-sentence of s1 matched to s2 and then detect the sub- sentence of s2 matched to s1. For each word wi in s1 we search for its match wj in s2 according to matching features. S1: …

Content truncated to 500 words