DIT: Summarisation and Semantic Expansion in Evaluating Semantic Similarity

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 230–234, Dublin, Ireland, August 23-24, 2014. DIT: Summarisation and Semantic Expansion in Evaluating Semantic Similarity Magdalena Kacmajor IBM Technology Campus Dublin Software Lab Ireland magdalena.kacmajor@ie.ibm.com John D. Kelleher Applied Intelligence Research Centre Dublin Institute of Technology Ireland john.d.kelleher@dit.ie Abstract This paper describes an approach to im- plementing a tool for evaluating seman- tic similarity. We investigated the poten- tial benefits of (1) using text summarisa- tion to narrow down the comparison to the most important concepts in both texts, and (2) leveraging WordNet information to in- crease usefulness of cosine comparisons of short texts. In our experiments, text summarisation using a graph-based algo- rithm did not prove to be helpful. Se- mantic and lexical expansion based upon word relationships defined in WordNet in- creased the agreement of cosine similarity values with human similarity judgements. 1 Introduction This paper describes a system that addresses the problem of assessing semantic similarity between two different-sized texts. The system has been ap- plied to SemEval-2014 Task 3, Cross-Level Se- mantic Similarity (Jurgens et al, 2014). The appli- cation is limited to a single comparison type, that is, paragraph to sentence. The general approach taken can be charac- terised as text summarisation followed by a pro- cess of semantic expansion and finally similarity computation using cosine similarity. The rationale for applying summarisation is to focus the comparison on the most important ele- ments of the text by selecting key words to be used in the similarity comparison. This summarisation approach is based on the assumption that if sum- mary of a paragraph is similar to the summary sen- tence paired with the paragraph in the task dataset, then the original paragraph and sentence pair must This work is licensed under a Creative Commons Attribution 4.0 International Licence. Page numbers and proceedings footer are added by the organisers. Licence details: http: //creativecommons.org/licenses/by/4.0/ have been similar and so should receive a high similarity rating. The subsequent semantic expansion is intended to counteract the problem arising from the small size of both compared text units. The similarity metric used by the system is essentially a func- tion of word overlap. However, because both the paragraphs and sentences being compared are rel- atively short, the probability of a word overlap - even between semantically similar texts - is quite small. Therefore prior to estimating the similarity between the texts we extend the word vectors cre- ated by the summarisation process with the syn- onyms and other words semantically and lexically related to the words occurring in the text. By using cosine similarity measure, we nor- malize the lengths of word vectors representing different-sized documents (paragraphs and sen- tences). The rest of the paper is organized as follows: section 2 describes the components of the sys- tem in more detail; section 3 describes parame- ters used in the experiments we conducted and presents our results; and section 4 provides con- cluding remarks. 2 System Description 2.1 Overview There …

Content truncated to 500 words