Twitter Paraphrase Identification with Simple Overlap Features and SVMs

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 64–69, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics ASOBEK: Twitter Paraphrase Identification with Simple Overlap Features and SVMs Asli Eyecioglu and Bill Keller Department of Informatics The University of Sussex Brighton, UK A.Eyecioglu@sussex.ac.uk, billk@sussex.ac.uk Abstract We present an approach to identifying Twitter paraphrases using simple lexical over- lap features. The work is part of ongoing re- search into the applicability of knowledge- lean techniques to paraphrase identification. We utilize features based on overlap of word and character n-grams and train support vector machine (SVM). Our results demonstrate that character and word level overlap features in combination can give performance compara- ble to methods employing more sophisticated NLP processing tools and external resources. We achieve the highest F-score for identifying paraphrases on the Twitter Paraphrase Corpus as part of the SemEval-2015 Task1. 1 Introduction This paper presents an approach to identifying Twitter paraphrase pairs using lexical overlap fea- tures. Paraphrase identification (PI) may be de- fined as “the task of deciding whether two given text fragments have the same meaning” (Lintean & Rus 2011). Methods for identifying paraphrases thus take a pair of texts and make a binary judg- ment. The PI task has practical importance in the Natural Language Processing (NLP) community because of the pervasive problem of linguistic var- iation. Accurate methods for PI should help im- prove the performance of NLP systems that would seem to require language understanding. This in- cludes key applications such as question answer- ing, information retrieval and machine translation, amongst others. Acquired paraphrases have been shown to improve the performance of Statistical Machine Translation (SMT) systems, for example (Callison-Burch et al. 2006, Owczarzak et al., 2006; Madnani et al., 2007) Many researchers on PI make use of existing NLP tools and other resources to identify para- phrases. For example, Duclaye et al., (2002) ex- ploits the NLP tools of a question answering system for reformulating rules to identify para- phrases. Other researchers (Finch et al 2005, Mihalcea et al 2006, Fernando & Stevenson 2008, Malakasiotis 2009, Das & Smith 2009) have em- ployed lexical semantic similarity information based on resources such as WordNet (Miller, 1995). Although the PI task aims to identify sentenc- es that are semantically equivalent, a number of researchers have shown that classifiers trained on lexical overlap features may achieve relatively high accuracy. Good performance is achieved without the use of knowledge-based semantic fea- tures or other external knowledge sources such as parallel corpora (Lintean & Rus 2011, Blacoe & Lapata, 2012). We consider methods as knowledge-lean if they make use of just the text at hand and avoid the use of external processing tools and other resources. Knowledge-lean PI methods may thus employ shallow overlap measures based on lexical items or n-grams, but they might also make use of distributional techniques where these are based on simple text statistics. The work described here is part of ongoing research that is investigating the extent to …

Content truncated to 500 words