KLUE-CORE: A regression model of semantic textual similarity

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 181–186, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics KLUE-CORE: A regression model of semantic textual similarity Paul Greiner and Thomas Proisl and Stefan Evert and Besim Kabashi Friedrich-Alexander-Universität Erlangen-Nürnberg Department Germanistik und Komparatistik Professur für Korpuslinguistik Bismarckstr. 6 91054 Erlangen, Germany {paul.greiner,thomas.proisl,stefan.evert,besim.kabashi}@fau.de Abstract This paper describes our system entered for the *SEM 2013 shared task on Semantic Textual Similarity (STS). We focus on the core task of predicting the semantic textual similarity of sentence pairs. The current system utilizes machine learn- ing techniques trained on semantic similarity ratings from the *SEM 2012 shared task; it achieved rank 20 out of 90 submissions from 35 different teams. Given the simple nature of our approach, which uses only WordNet and unannotated corpus data as external resources, we consider this a remarkably good result, mak- ing the system an interesting tool for a wide range of practical applications. 1 Introduction The *SEM 2013 shared task on Semantic Textual Similarity (Agirre et al., 2013) required participants to implement a software system that is able to pre- dict the semantic textual similarity (STS) of sentence pairs. Being able to reliably measure semantic simi- larity can be beneficial for many applications, e.g. in the domains of MT evaluation, information extrac- tion, question answering, and summarization. For the shared task, STS was measured on a scale ranging from 0 (indicating no similarity at all) to 5 (semantic equivalence). The system predictions were evaluated against manually annotated data. 2 Description of our approach Our system KLUE-CORE uses two approaches to estimate STS between pairs of sentences: a distri- butional bag-of-words model inspired by Schütze (1998), and a simple alignment model that links each word in one sentence to the semantically most similar word in the other sentence. For the alignment model, word similarities were obtained from WordNet (using a range of state-of-the-art path-based similarity mea- sures) and from two distributional semantic models (DSM). All similarity scores obtained in this way were passed to a ridge regression learner in order to obtain a final STS score. The predictions for new sentence pairs were then transformed to the range [0,5], as required by the task definition. 2.1 The training data We trained our system on manually annotated sen- tence pairs from the STS task at SemEval 2012 (Agirre et al., 2012). Pooling the STS 2012 training and test data, we obtained 5 data sets from differ- ent domains, comprising a total of 5343 sentence pairs annotated with a semantic similarity score in the range [0,5]. The data sets are paraphrase sen- tence pairs (MSRpar), sentence pairs from video de- scriptions (MSRvid), MT evaluation sentence pairs (MTnews and MTeuroparl), and glosses from two different lexical semantic resources (OnWN). All sentence pairs were pre-processed with Tree- Tagger (Schmid, 1995)1 for part-of-speech annota- tion and lemmatization. 1http://www.ims.uni-stuttgart.de/forschung/ ressourcen/werkzeuge/treetagger.html 181 2.2 Similarity on word level Our alignment model (Sec. 2.3.1) …

Content truncated to 500 words