A Continuously Growing Dataset of Sentential Paraphrases

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1224–1234 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics A Continuously Growing Dataset of Sentential Paraphrases Wuwei Lan1, Siyu Qiu2, Hua He3, Wei Xu1 1 Department of Computer Science and Engineering Ohio State University {lan.105, xu.1265}@osu.edu 2 Computer and Information Science Department University of Pennsylvania siqiu@seas.upenn.edu 3 Department of Computer Science University of Maryland, College Park huah@umd.edu Abstract A major challenge in paraphrase research is the lack of parallel corpora. In this pa- per, we present a new method to collect large-scale sentential paraphrases from Twitter by linking tweets through shared URLs. The main advantage of our method is its simplicity, as it gets rid of the classi- fier or human in the loop needed to select data before annotation and subsequent ap- plication of paraphrase identification algo- rithms in the previous work. We present the largest human-labeled paraphrase cor- pus to date of 51,524 sentence pairs and the first cross-domain benchmarking for automatic paraphrase identification. In ad- dition, we show that more than 30,000 new sentential paraphrases can be easily and continuously captured every month at ∼70% precision, and demonstrate their utility for downstream NLP tasks through phrasal paraphrase extraction. We make our code and data freely available.1 1 Introduction A paraphrase is a restatement of meaning using different expressions (Bhagat and Hovy, 2013). It is a fundamental semantic relation in human lan- guage, as formalized in the Meaning-Text linguis- tic theory which defines meaning as ‘invariant of paraphrases’ (Mili´cevi´c, 2006). Researchers have shown benefits of using paraphrases in a wide range of applications (Madnani and Dorr, 2010), including question answering (Fader et al., 2013), semantic parsing (Berant and Liang, 2014), in- formation extraction (Sekine, 2006; Zhang et al., 1The code and data can be obtained from the first and last author’s websites. 2015), machine translation (Mehdizadeh Seraj et al., 2015), textual entailment (Dagan et al., 2006; Bjerva et al., 2014; Marelli et al., 2014; Iza- dinia et al., 2015), vector semantics (Faruqui et al., 2015; Wieting et al., 2015), and semantic textual similarity (Agirre et al., 2015; Li and Srikumar, 2016). Studying paraphrases in Twitter can also help track unfolding events (Vosoughi and Roy, 2016) or the spread of information (Bakshy et al., 2011) on social networks. In this paper, we address a major challenge in paraphrase research — the lack of parallel corpora. There are only two publicly available datasets of naturally occurring sentential para- phrases and non-paraphrases:2 the MSRP corpus derived from clustered news articles (Dolan and Brockett, 2005) and the PIT-2015 corpus from Twitter trending topics (Xu et al., 2014, 2015). Our goal is not only to create a new annotated paraphrase corpus, but to identify a new data source and method that can narrow down the search space of paraphrases without using the classifier-biased or human-in-the-loop data selec- tion as in MSRP and PIT-2015. This is so that sen- tential paraphrases can be conveniently and con- tinuously harvested in …

Content truncated to 500 words