UNT: A Supervised Synergistic Approach to Semantic Text Similarity

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 635–642, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics UNT: A Supervised Synergistic Approach to Semantic Text Similarity Carmen Banea, Samer Hassan, Michael Mohler, Rada Mihalcea University of North Texas Denton, TX, USA {CarmenBanea,SamerHassan,MichaelMohler}@my.unt.edu, rada@cs.unt.edu Abstract This paper presents the systems that we par- ticipated with in the Semantic Text Similar- ity task at SEMEVAL 2012. Based on prior research in semantic similarity and related- ness, we combine various methods in a ma- chine learning framework. The three varia- tions submitted during the task evaluation pe- riod ranked number 5, 9 and 14 among the 89 participating systems. Our evaluations show that corpus-based methods display a more ro- bust behavior on the training data, yet com- bining a variety of methods allows a learning algorithm to achieve a superior decision than that achievable by any of the individual parts. 1 Introduction Measures of text similarity have been used for a long time in applications in natural language pro- cessing and related areas. One of the earliest ap- plications of text similarity is perhaps the vector- space model used in information retrieval, where the document most relevant to an input query is deter- mined by ranking documents in a collection in re- versed order of their similarity to the given query (Salton and Lesk, 1971). Text similarity has also been used for relevance feedback and text classifi- cation (Rocchio, 1971), word sense disambiguation (Lesk, 1986; Schutze, 1998), and more recently for extractive summarization (Salton et al., 1997), and methods for automatic evaluation of machine trans- lation (Papineni et al., 2002) or text summarization (Lin and Hovy, 2003). Measures of text similarity were also found useful for the evaluation of text co- herence (Lapata and Barzilay, 2005). Earlier work on this task has primarily focused on simple lexical matching methods, which produce a similarity score based on the number of lexical units that occur in both input segments. Improvements to this simple method have considered stemming, stop-word removal, part-of-speech tagging, longest subsequence matching, as well as various weight- ing and normalization factors (Salton and Buckley, 1997). While successful to a certain degree, these lexical similarity methods cannot always identify the semantic similarity of texts. For instance, there is an obvious similarity between the text segments I own a dog and I have an animal, but most of the current text similarity metrics will fail in identifying any kind of connection between these texts. More recently, researchers have started to con- sider the possibility of combining the large number of word-to-word semantic similarity measures (e.g., (Jiang and Conrath, 1997; Leacock and Chodorow, 1998; Lin, 1998; Resnik, 1995)) within a semantic similarity method that works for entire texts. The methods proposed to date in this direction mainly consist of either bipartite-graph matching strate- gies that aggregate word-to-word similarity into a text similarity score (Mihalcea et al., 2006; Islam and Inkpen, 2009; Hassan and Mihalcea, 2011; Mohler et al., 2011), or data-driven methods …

Content truncated to 500 words