Extracting Lexically Divergent Paraphrases from Twitter

Extracting Lexically Divergent Paraphrases from Twitter Wei Xu1, Alan Ritter2, Chris Callison-Burch1, William B. Dolan3 and Yangfeng Ji4 1 University of Pennsylvania, Philadelphia, PA, USA {xwe, ccb}@cis.upenn.edu 2 The Ohio State University, Columbus, OH, USA ritter.1492@osu.edu 3 Microsoft Research, Redmond, WA, USA billdol@microsoft.com 4 Georgia Institute of Technology, Atlanta, GA, USA jiyfeng@gatech.edu Abstract We present MULTIP (Multi-instance Learn- ing Paraphrase Model), a new model suited to identify paraphrases within the short mes- sages on Twitter. We jointly model para- phrase relations between word and sentence pairs and assume only sentence-level annota- tions during learning. Using this principled la- tent variable model alone, we achieve the per- formance competitive with a state-of-the-art method which combines a latent space model with a feature-based supervised classifier. Our model also captures lexically divergent para- phrases that differ from yet complement previ- ous methods; combining our model with pre- vious work significantly outperforms the state- of-the-art. In addition, we present a novel an- notation methodology that has allowed us to crowdsource a paraphrase corpus from Twit- ter. We make this new dataset available to the research community. 1 Introduction Paraphrases are alternative linguistic expressions of the same or similar meaning (Bhagat and Hovy, 2013). Twitter engages millions of users, who nat- urally talk about the same topics simultaneously and frequently convey similar meaning using diverse linguistic expressions. The unique characteristics of this user-generated text presents new challenges and opportunities for paraphrase research (Xu et al., 2013b; Wang et al., 2013). For many applications, like automatic summarization, first story detection (Petrovi´c et al., 2012) and search (Zanzotto et al., 2011), it is crucial to resolve redundancy in tweets (e.g. oscar nom’d doc ↔ Oscar-nominated docu- mentary). In this paper, we investigate the task of determin- ing whether two tweets are paraphrases. Previous work has exploited a pair of shared named entities to locate semantically equivalent patterns from re- lated news articles (Shinyama et al., 2002; Sekine, 2005; Zhang and Weld, 2013). But short sentences in Twitter do not often mention two named entities (Ritter et al., 2012) and require nontrivial general- ization from named entities to other words. For ex- ample, consider the following two sentences about basketball player Brook Lopez from Twitter: ◦ That boy Brook Lopez with a deep 3 ◦ brook lopez hit a 3 and i missed it Although these sentences do not have many words in common, the identical word “3” is a strong indicator that the two sentences are paraphrases. We therefore propose a novel joint word-sentence approach, incorporating a multi-instance learning assumption (Dietterich et al., 1997) that two sen- tences under the same topic (we highlight topics in bold) are paraphrases if they contain at least one word pair (we call it an anchor and highlight with underscores; the words in the anchor pair need not be identical) that is indicative of sentential para- phrase. This at-least-one-anchor assumption might be ineffective for long or randomly paired sentences, but holds up better for short sentences that are tem- porally and …

Content truncated to 500 words