ROB: Using Semantic Meaning to Recognize Paraphrases

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 40–44, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics ROB: Using Semantic Meaning to Recognize Paraphrases Rob van der Goot University of Groningen r.van.der.goot@rug.nl Gertjan van Noord University of Groningen g.j.m.van.noord@rug.nl Abstract Paraphrase recognition is the task of iden- tifying whether two pieces of natural lan- guage represent similar meanings. This pa- per describes a system participating in the shared task 1 of SemEval 2015, which is about paraphrase detection and semantic similarity in twitter. Our approach is to exploit se- mantically meaningful features to detect para- phrases. An existing state-of-the-art model for predicting semantic similarity is adapted to this task. A wide variety of features is used, ranging from different types of models, to lexical over- lap and synset overlap. A maximum entropy classifier is then trained on these features. In addition to the detection of paraphrases, a sim- ilarity score is also predicted, using the proba- bilities of the classifier. To improve the results, normalization is used as preprocessing step. Our final system achieves a F1 score of 0.620 (10th out of 18 teams), and a Pearson correla- tion of 0.515 (6th out of 13 teams). 1 Introduction A good paraphrase detection system can be useful in many natural language processing tasks, like search- ing, translating or summarization. For clean texts, F1 scores as high as 0.84 have been reported on para- phrase detection (Madnani et al., 2012). However, previous research focused almost solely on clean text. Thanks to the Twitter Paraphrase Cor- pus (Xu et al., 2014), this has now changed. Car- rying out this task on noisy texts is a new chal- lenge. The abundant availability of social media data and the high redundancy that naturally exists in this data makes this task highly relevant (Zanzotto et al., 2011). Our approach is based on the model described by Bjerva et al. (2014). This model has proved to achieve state-of-the-art results at predicting se- mantic similarity (Marelli et al., 2014). It is based on overlaps of semantically meaningful properties of sentences. A random forest regression model (Breiman, 2001) combines these features to predict a semantic similarity score. We rely heavily on the as- sumption that semantically meaningful features can also be used to identify paraphrases. The features of the existing system are also used in the new system. However, the old system used a regression model, while the new task demands class- based output. Hence, the machine learning model model is changed to a maximum entropy model. 2 Data The Twitter Paraphrase Corpus consists of two dis- tinct parts, the training data differs significantly from the test data. The 17,790 tweet pairs for training are collected between April 24th and May 3rd, 2014. These tweets are selected based on the trending topics of that period. Annotation of the training data is done by human annotators from Amazon Mechan- ical Turk. Every sentence pair is annotated by 5 dif- ferent annotators, resulting in a …

Content truncated to 500 words