Monolingual Distributional Similarity for Text-to-Text Generation

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 256–264, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Monolingual Distributional Similarity for Text-to-Text Generation Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University Baltimore, MD 21218, USA Abstract Previous work on paraphrase extraction and application has relied on either parallel datasets, or on distributional similarity met- rics over large text corpora. Our approach combines these two orthogonal sources of in- formation and directly integrates them into our paraphrasing system’s log-linear model. We compare different distributional similar- ity feature-sets and show significant improve- ments in grammaticality and meaning reten- tion on the example text-to-text generation task of sentence compression, achieving state- of-the-art quality. 1 Introduction A wide variety of applications in natural language processing can be cast in terms of text-to-text gen- eration. Given input in the form of natural lan- guage, a text-to-text generation system produces natural language output that is subject to a set of constraints. Compression systems, for instance, pro- duce shorter sentences. Paraphrases, i.e. differ- ing textual realizations of the same meaning, are a crucial components of text-to-text generation sys- tems, and have been successfully applied to tasks such as multi-document summarization (Barzilay et al., 1999; Barzilay, 2003), query expansion (An- ick and Tipirneni, 1999; Riezler et al., 2007), ques- tion answering (McKeown, 1979; Ravichandran and Hovy, 2002), sentence compression (Cohn and La- pata, 2008; Zhao et al., 2009), and simplification (Wubben et al., 2012). Paraphrase collections for text-to-text generation have been extracted from a variety of different cor- pora. Several approaches rely on bilingual paral- lel data (Bannard and Callison-Burch, 2005; Zhao et al., 2008; Callison-Burch, 2008; Ganitkevitch et al., 2011), while others leverage distributional meth- ods on monolingual text corpora (Lin and Pantel, 2001; Bhagat and Ravichandran, 2008). So far, how- ever, only preliminary studies have been undertaken to combine the information from these two sources (Chan et al., 2011). In this paper, we describe an extension of Gan- itkevitch et al. (2011)’s bilingual data-based ap- proach. We augment the bilingually-sourced para- phrases using features based on monolingual distri- butional similarity. More specifically: • We show that using monolingual distributional similarity features improves paraphrase quality beyond what we can achieve with features esti- mated from bilingual data. • We define distributional similarity for para- phrase patterns that contain constituent-level gaps, e.g. sim(one JJ instance of NP, a JJ case of NP). This generalizes over distributional similarity for contiguous phrases. • We compare different types of monolingual distributional information and show that they can be used to achieve significant improve- ments in grammaticality. • Finally, we compare our method to several strong baselines on the text-to-text generation task of sentence compression. Our method shows state-of-the-art results, beating a purely bilingually sourced paraphrasing system. 256 ... ihre Pläne würden their plans would ... ohne aufzugeben without langfristigen in the long term langfristigen long-term ... ... ... ... Pläne plans seine …

Content truncated to 500 words