ATA-Sem: Chunk-based Determination of Semantic Text Similarity

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 547–551, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics ATA-Sem: Chunk-based Determination of Semantic Text Similarity Demetrios Glinos Advanced Text Analytics, LLC Orlando, Florida, USA demetrios.glinos@advancedtextanalytics.com Abstract This paper describes investigations into using syntactic chunk information as the basis for determining the similarity of candidate texts at the semantic level. Two approaches were con- sidered. The first was a corpus-based method that extracted lexical and semantic features from pairs of chunks from each sentence that were associated through a chunk alignment algorithm. The features were used as input to a classifier trained on the same features ex- tracted from a corpus of gold standard training data. The second approach involved breadth- first chunk association and the application of a rule-based scoring algorithm. Both approaches were evaluated against the test data for the SemEval 2012 Semantic Text Similarity task. The results show that the rule-based chunk approach is superior. 1 Introduction The task of determining whether two texts are sim- ilar in some sense has important applications in the field of natural language processing, including but not limited to document summarization (Evans, et al., 2005), plagiarism detection (Barrón-Cedeño, et al., 2009) and large corpus document retrieval (Charikar, 2002). While textual similarity can be performed at the purely surface lexical level, as in the “simhash” clustering method described in (Moulton, 2010), similarity also applies at the semantic level, where conceptually similar texts may nevertheless be en- tirely dissimilar at the surface lexical level. For example, the phrases “restrict or confine” and “place limits on (extent or access)” share no words or morphological roots, yet mean very nearly the same thing at the semantic level. The Semantic Textual Similarity (STS) task (Task #6) at SemEval-2012 (Agirre, et al., 2012) provided a forum for exploring these issues by fur- nishing training and evaluation data, and also a common standard for describing degrees of simi- larity, shown in Table 1. Score Description 5 The two sentences are completely equiva- lent, as they mean the same thing. 4 The two sentences are mostly equivalent, but some unimportant details differ. 3 The two sentences are roughly equivalent, but some important information dif- fers/missing. 2 The two sentences are not equivalent, but share some details. 1 The two sentences are not equivalent, but are on the same topic. 0 The two sentences are on different topics. Table 1. STS similarity scoring standard. Our corpus-based chunk similarity method par- ticipated in the formal STS evaluation. Our rule- based method was completed after the submittal date, but we report on it here because the method does not involve training on a corpus, nor any pa- rameter tuning, and because it significantly outper- formed the corpus-based method. The remainder of this paper is organized as fol- lows. In the next section, we describe the common processing components for both methods. Section 3 then presents the corpus-based chunk method, followed in Section 4 by a discussion of the rule- based …

Content truncated to 500 words