NTNU-CORE: Combining strong features for semantic similarity

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 66–73, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics NTNU-CORE: Combining strong features for semantic similarity Erwin Marsi, Hans Moen, Lars Bungum, Gleb Sizov, Bj¨orn Gamb¨ack, Andr´e Lynum Norwegian University of Science and Technology Department of Computer and Information and Science Sem Sælands vei 7-9 NO-7491 Trondheim, Norway {emarsi,hansmoe,bungum,sizov,gamback,andrely}@idi.ntnu.no Abstract The paper outlines the work carried out at NTNU as part of the *SEM’13 shared task on Semantic Textual Similarity, using an ap- proach which combines shallow textual, dis- tributional and knowledge-based features by a support vector regression model. Feature sets include (1) aggregated similarity based on named entity recognition with WordNet and Levenshtein distance through the calcula- tion of maximum weighted bipartite graphs; (2) higher order word co-occurrence simi- larity using a novel method called “Multi- sense Random Indexing”; (3) deeper seman- tic relations based on the RelEx semantic dependency relationship extraction system; (4) graph edit-distance on dependency trees; (5) reused features of the TakeLab and DKPro systems from the STS’12 shared task. The NTNU systems obtained 9th place overall (5th best team) and 1st place on the SMT data set. 1 Introduction Intuitively, two texts are semantically similar if they roughly mean the same thing. The task of formally establishing semantic textual similarity clearly is more complex. For a start, it implies that we have a way to formally represent the intended meaning of all texts in all possible contexts, and furthermore a way to measure the degree of equivalence between two such representations. This goes far beyond the state-of-the-art for arbitrary sentence pairs, and sev- eral restrictions must be imposed. The Semantic Textual Similarity (STS) task (Agirre et al., 2012, 2013) limits the comparison to isolated sentences only (rather than complete texts), and defines sim- ilarity of a pair of sentences as the one assigned by human judges on a 0–5 scale (with 0 implying no relation and 5 complete semantic equivalence). It is unclear, however, to what extent two judges would agree on the level of similarity between sentences; Agirre et al. (2012) report figures on the agreement between the authors themselves of about 87–89%. As in most language processing tasks, there are two overall ways to measure sentence similarity, ei- ther by data-driven (distributional) methods or by knowledge-driven methods; in the STS’12 task the two approaches were used nearly equally much. Distributional models normally measure similarity in terms of word or word co-occurrence statistics, or through concept relations extracted from a corpus. The basic strategy taken by NTNU in the STS’13 task was to use something of a “feature carpet bomb- ing approach” in the way of first automatically ex- tracting as many potentially useful features as possi- ble, using both knowledge and data-driven methods, and then evaluating feature combinations on the data sets provided by the organisers of the shared task. To this end, four different types of features were extracted. …

Content truncated to 500 words