DeepPurple: Estimating Sentence Semantic Similarity using N-gram Regression Models and Web Snippets

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 565–570, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics DeepPurple: Estimating Sentence Semantic Similarity using N-gram Regression Models and Web Snippets Nikos Malandrakis, Elias Iosif, Alexandros Potamianos Department of ECE, Technical University of Crete, 73100 Chania, Greece [nmalandrakis,iosife,potam]@telecom.tuc.gr Abstract We estimate the semantic similarity between two sentences using regression models with features: 1) n-gram hit rates (lexical matches) between sentences, 2) lexical semantic sim- ilarity between non-matching words, and 3) sentence length. Lexical semantic similarity is computed via co-occurrence counts on a cor- pus harvested from the web using a modified mutual information metric. State-of-the-art re- sults are obtained for semantic similarity com- putation at the word level, however, the fusion of this information at the sentence level pro- vides only moderate improvement on Task 6 of SemEval’12. Despite the simple features used, regression models provide good perfor- mance, especially for shorter sentences, reach- ing correlation of 0.62 on the SemEval test set. 1 Introduction Recently, there has been significant research activ- ity on the area of semantic similarity estimation motivated both by abundance of relevant web data and linguistic resources for this task. Algorithms for computing semantic textual similarity (STS) are relevant for a variety of applications, including in- formation extraction (Szpektor and Dagan, 2008), question answering (Harabagiu and Hickl, 2006) and machine translation (Mirkin et al., 2009). Word- or term-level STS (a special case of sentence level STS) has also been successfully applied to the prob- lem of grammar induction (Meng and Siu, 2002) and affective text categorization (Malandrakis et al., 2011). In this work, we built on previous research on word-level semantic similarity estimation to de- sign and implement a system for sentence-level STS for Task6 of the SemEval’12 campaign. Semantic similarity between words can be re- garded as the graded semantic equivalence at the lexeme level and is tightly related with the tasks of word sense discovery and disambiguation (Agirre and Edmonds, 2007). Metrics of word semantic sim- ilarity can be divided into: (i) knowledge-based met- rics (Miller, 1990; Budanitsky and Hirst, 2006) and (ii) corpus-based metrics (Baroni and Lenci, 2010; Iosif and Potamianos, 2010). When more complex structures, such as phrases and sentences, are considered, it is much harder to estimate semantic equivalence due to the non- compositional nature of sentence-level semantics and the exponential explosion of possible interpre- tations. STS is closely related to the problems of paraphrasing, which is bidirectional and based on semantic equivalence (Madnani and Dorr, 2010) and textual entailment, which is directional and based on relations between semantics (Dagan et al., 2006). Related methods incorporate measurements of sim- ilarity at various levels: lexical (Malakasiotis and Androutsopoulos, 2007), syntactic (Malakasiotis, 2009; Zanzotto et al., 2009), and semantic (Rinaldi et al., 2003; Bos and Markert, 2005). Measures from machine translation evaluation are often used to evaluate lexical level approaches (Finch et al., 2005; Perez and Alfonseca, 2005), including BLEU (Papineni et al., 2002), a metric based on word n- gram …

Content truncated to 500 words