ECNUCS: Measuring Short Text Semantic Equivalence Using Multiple Similarity Measurements

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 124–131, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics ECNUCS: Measuring Short Text Semantic Equivalence Using Multiple Similarity Measurements Tian Tian ZHU Department of Computer Science and Technology East China Normal University 51111201046@student.ecnu.edu.cn Man LAN∗ Department of Computer Science and Technology East China Normal University mlan@cs.ecnu.edu.cn Abstract This paper reports our submissions to the Semantic Textual Similarity (STS) task in ∗SEM Shared Task 2013. We submitted three Support Vector Regression (SVR) systems in core task, using 6 types of similarity mea- sures, i.e., string similarity, number similar- ity, knowledge-based similarity, corpus-based similarity, syntactic dependency similarity and machine translation similarity. Our third sys- tem with different training data and different feature sets for each test data set performs the best and ranks 35 out of 90 runs. We also sub- mitted two systems in typed task using string based measure and Named Entity based mea- sure. Our best system ranks 5 out of 15 runs. 1 Introduction The task of semantic textual similarity (STS) is to measure the degree of semantic equivalence between two sentences, which plays an increasingly impor- tant role in natural language processing (NLP) ap- plications. For example, in text categorization (Yang and Wen, 2007), two documents which are more similar are more likely to be grouped in the same class. In information retrieval (Sahami and Heil- man, 2006), text similarity improves the effective- ness of a semantic search engine by providing in- formation which holds high similarity with the input query. In machine translation (Kauchak and Barzi- lay, 2006), sentence similarity can be applied for automatic evaluation of the output translation and the reference translations. In question answering (Mohler and Mihalcea, 2009), once the question and the candidate answers are treated as two texts, the answer text which has a higher relevance with the question text may have higher probability to be the right one. The STS task in ∗SEM Shared Task 2013 consists of two subtasks, i.e., core task and typed task, and we participate in both of them. The core task aims to measure the semantic similarity of two sentences, resulting in a similarity score which ranges from 5 (semantic equivalence) to 0 (no relation). The typed task is a pilot task on typed-similarity between semi- structured records. The types of similarity to be measured include location, author, people involved, time, events or actions, subject and description as well as the general similarity of two texts (Agirre et al., 2013). In this work we present a Support Vector Re- gression (SVR) system to measure sentence seman- tic similarity by integrating multiple measurements, i.e., string similarity, knowledge based similarity, corpus based similarity, number similarity and ma- chine translation metrics. Most of these similari- ties are borrowed from previous work, e.g., (B¨ar et al., 2012), (ˇSaric et al., 2012) and (de Souza et al., 2012). We also propose a novel syntactic depen- dency similarity. …

Content truncated to 500 words