UMBC_EBIQUITY-CORE: Semantic Textual Similarity Systems

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 44–52, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics UMBC EBIQUITY-CORE: Semantic Textual Similarity Systems Lushan Han, Abhay Kashyap, Tim Finin Computer Science and Electrical Engineering University of Maryland, Baltimore County Baltimore MD 21250 {lushan1,abhay1,finin}@umbc.edu James Mayfield and Jonathan Weese Human Language Technology Center of Excellence Johns Hopkins University Baltimore MD 21211 mayfield@jhu.edu, jonny@cs.jhu.edu Abstract We describe three semantic text similarity systems developed for the *SEM 2013 STS shared task and the results of the correspond- ing three runs. All of them shared a word sim- ilarity feature that combined LSA word sim- ilarity and WordNet knowledge. The first, which achieved the best mean score of the 89 submitted runs, used a simple term alignment algorithm augmented with penalty terms. The other two runs, ranked second and fourth, used support vector regression models to combine larger sets of features. 1 Introduction Measuring semantic text similarity has been a re- search subject in natural language processing, infor- mation retrieval and artificial intelligence for many years. Previous efforts have focused on compar- ing two long texts (e.g., for document classification) or a short text with a long text (e.g., Web search), but there are a growing number of tasks requiring computing the semantic similarity between two sen- tences or other short text sequences. They include paraphrase recognition (Dolan et al., 2004), Twitter tweets search (Sriram et al., 2010), image retrieval by captions (Coelho et al., 2004), query reformula- tion (Metzler et al., 2007), automatic machine trans- lation evaluation (Kauchak and Barzilay, 2006) and schema matching (Han et al., 2012). There are three predominant approaches to com- puting short text similarity. The first uses informa- tion retrieval’s vector space model (Meadow, 1992) in which each text is modeled as a “bag of words” and represented using a vector. The similarity be- tween two texts is then computed as the cosine similarity of the vectors. A variation on this ap- proach leverages web search results (e.g., snip- pets) to provide context for the short texts and en- rich their vectors using the words in the snippets (Sahami and Heilman, 2006). The second approach is based on the assumption that if two sentences or other short text sequences are semantically equiva- lent, we should be able to align their words or ex- pressions. The alignment quality can serve as a similarity measure. This technique typically pairs words from the two texts by maximizing the sum- mation of the word similarity of the resulting pairs (Mihalcea et al., 2006). The third approach com- bines different measures and features using machine learning models. Lexical, semantic and syntactic features are computed for the texts using a variety of resources and supplied to a classifier, which then assigns weights to the features by fitting the model to training data (Saric et al., 2012). For evaluating different approaches, the 2013 Se- mantic Textual Similarity (STS) task asked …

Content truncated to 500 words