Soft Cardinality: A Parameterized Similarity Function for Text Comparison

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 449–453, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Soft Cardinality: A Parameterized Similarity Function for Text Comparison Sergio Jimenez Universidad Nacional de Colombia, Bogota, Ciudad Universitaria edificio 453, oficina 220 sgjimenezv@unal.edu.co Claudia Becerra Universidad Nacional de Colombia, Bogota cjbecerrac@unal.edu.co Alexander Gelbukh CIC-IPN Av. Juan Dios Bátiz, Av. Mendizábal, Col. Nueva Industrial Vallejo, CP 07738, DF, México gelbukh@gelbukh.com Abstract We present an approach for the construction of text similarity functions using a parameterized resem- blance coefficient in combination with a softened cardinality function called soft cardinality. Our ap- proach provides a consistent and recursive model, varying levels of granularity from sentences to char- acters. Therefore, our model was used to compare sentences divided into words, and in turn, words di- vided into q-grams of characters. Experimentally, we observed that a performance correlation func- tion in a space defined by all parameters was rel- atively smooth and had a single maximum achiev- able by “hill climbing.” Our approach used only sur- face text information, a stop-word remover, and a stemmer to tackle the semantic text similarity task 6 at SEMEVAL 2012. The proposed method ranked 3rd (average), 5th (normalized correlation), and 15th (aggregated correlation) among 89 systems submit- ted by 31 teams. 1 Introduction Similarity is the intrinsic ability of humans and some animals to balance commonalities and differences when comparing objects that are not identical. Although there is no direct evidence of how this process works in liv- ing organisms, some models have been proposed from the cognitive perspective (Sjöberg, 1972; Tversky, 1977; Navarro and Lee, 2004). On the other hand, several simi- larity models have been proposed in mathematics, statis- tics, and computer science among other fields. Particu- larly in AI, similarity measures play an important role in the construction of intelligent systems that are required to exhibit behavior similar to humans. For instance, in the field of natural language processing, text similarity functions provide estimates of the human similarity judg- ments related to language. In this paper, we combine el- ements from the perspective of cognitive psychology and computer science to propose a model for building simi- larity functions suitable for the task of semantic text sim- ilarity. We identify four main families of text similarity func- tions: i) resemblance coefficients based on sets (e.g. Jac- card’s (1901) and Dice’s (1945) coefficients) ii) functions in metric spaces (e.g. cosine tf-idf similarity (Salton et al., 1975)); iii) the edit distance family of measures (e.g. Levenstein (1966) distance, LCS (Hirschberg, 1977)); and iv) hybrid approaches ((Monge and Elkan, 1996; Co- hen et al., 2003; Corley and Mihalcea, 2005; Jimenez et al., 2010)). All of these measures use a subdivision of the texts in different granularity levels, such as q-grams of words, words, q-grams of characters, syllables, and characters. Among hybrid approaches, Monge-Elkan’s measure and soft cardinality methods are recursive and can be used to build similarity functions at any arbitrary range of granularity. For instance, it is possible …

Content truncated to 500 words