FCICU: The Integration between Sense-Based Kernel and Surface-Based Methods to Measure Semantic Textual Similarity

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 154–158, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics FCICU: The Integration between Sense-Based Kernel and Surface- Based Methods to Measure Semantic Textual Similarity Basma Hassan Computer Science Department, Faculty of Computers and Information Fayoum University Fayoum, Egypt bhassan@fayoum.edu .eg Samir AbdelRahman Computer Science Department, Faculty of Computers and Information Cairo University Giza, Egypt s.abdelrahman@fci- cu.edu.eg Reem Bahgat Computer Science Department, Faculty of Computers and Information Cairo University Giza, Egypt r.bahgat@fci- cu.edu.eg Abstract This paper describes FCICU team participa- tion in SemEval 2015 for Semantic Textual Similarity challenge. Our main contribution is to propose a word-sense similarity method us- ing BabelNet relationships. In the English subtask challenge, we submitted three systems (runs) to assess the proposed method. In Run1, we used our proposed method coupled with a string kernel mapping function to cal- culate the textual similarity. In Run2, we used the method with a tree kernel function. In Run3, we averaged Run1 with a previously proposed surface-based approach as a kind of integration. The three runs are ranked 41st, 57th, and 20th of 73 systems, with mean corre- lation 0.702, 0.597, and 0.759 respectively. For the interpretable task, we submitted a modified version of Run1 achieving mean F1 0.846, 0.461, 0.722, and 0.44 for alignment, type, score, and score with type respectively. 1 Introduction Semantic Textual Similarity (STS) is the task of measuring the similarity between two text snippets according to their meaning. Human has an intrinsic ability to recognize the degree of similarity and difference between texts. Simulating the process of human judgment in computers is still an extremely difficult task and has recently drawn much atten- tion. STS is very important because a wide range of NLP applications such as information retrieval, question answering, machine translation, etc. rely heavily on this task. This paper describes our proposed STS systems by which we participated in two subtasks of STS task (Task2) at SemEval 2015, namely English STS and Interpretable STS. The former calculates a graded similarity score from 0 to 5 between two sentences (with 5 being the most similar), while the latter is a pilot subtask that requires aligning chunks of two sentences, describing what kind of relation exists between each pair of chunks, and a score for the similarity between the pair of chunks (Agirre et al., 2015). Sense or meaning of natural language text can be inferred from several linguistic concepts, in- cluding lexical, syntactic, and semantic knowledge of the language. Our approach employs those as- pects to calculate the similarity between senses of text constituents, phrases or words, relying mainly on BabelNet senses. The similarity between two text snippets is firstly calculated using kernel func- tions, which map a text snippet to the feature space based on a proposed word sense similarity method. Besides, the sense-based similarity score obtained is combined with a surface-based similarity score to study the consolidation impact in the STS task. The paper is organized as …

Content truncated to 500 words