University_Of_Sheffield: Two Approaches to Semantic Text Similarity

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 655–661, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics University Of Sheffield: Two Approaches to Semantic Text Similarity Sam Biggins, Shaabi Mohammed, Sam Oakley, Luke Stringer, Mark Stevenson and Judita Priess Department of Computer Science University of Sheffield Sheffield S1 4DP, UK {aca08sb, aca08sm, coa07so, aca08ls, r.m.stevenson, j.preiss}@shef.ac.uk Abstract This paper describes the University of Sheffield’s submission to SemEval-2012 Task 6: Semantic Text Similarity. Two approaches were developed. The first is an unsupervised technique based on the widely used vector space model and information from WordNet. The second method relies on supervised ma- chine learning and represents each sentence as a set of n-grams. This approach also makes use of information from WordNet. Results from the formal evaluation show that both ap- proaches are useful for determining the simi- larity in meaning between pairs of sentences with the best performance being obtained by the supervised approach. Incorporating infor- mation from WordNet also improves perfor- mance for both approaches. 1 Introduction This paper describes the University of Sheffield’s submission to SemEval-2012 Task 6: Semantic Text Similarity (Agirre et al., 2012). The task is con- cerned with determining the degree of semantic equivalence between a pair of sentences. Measuring the similarity between sentences is an important problem that is relevant to many areas of language processing, including the identification of text reuse (Seo and Croft, 2008; Bendersky and Croft, 2009), textual entailment (Szpektor et al., 2004; Zanzotto et al., 2009), paraphrase detection (Barzilay and Lee, 2003; Dolan et al., 2004), In- formation Extraction/Question Answering (Lin and Pantel, 2001; Stevenson and Greenwood, 2005), In- formation Retrieval (Baeza-Yates and Ribeiro-Neto, 1999), short answer grading (Pulman and Sukkarieh, 2005; Mohler and Mihalcea, 2009), recommenda- tion (Tintarev and Masthoff, 2006) and evaluation (Papineni et al., 2002; Lin, 2004). Many of the previous approaches to measuring the similarity between texts have relied purely on lexi- cal matching techniques, for example (Baeza-Yates and Ribeiro-Neto, 1999; Papineni et al., 2002; Lin, 2004). In these approaches the similarity of texts is computed as a function of the number of matching tokens, or sequences of tokens, they contain. How- ever, this approach fails to identify similarities when the same meaning is conveyed using synonymous terms or phrases (for example, “The dog sat on the mat” and “The hound sat on the mat”) or when the meanings of the texts are similar but not identical (for example, “The cat sat on the mat” and “A dog sat on the chair”). Significant amounts of previous work on text similarity have focussed on comparing the mean- ings of texts longer than a single sentence, such as paragraphs or documents (Baeza-Yates and Ribeiro- Neto, 1999; Seo and Croft, 2008; Bendersky and Croft, 2009). The size of these texts means that there is a reasonable amount of lexical items in each document that can be used to determine similarity and failing to identify connections between related terms may not be problematic. The situation is …

Content truncated to 500 words