Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment

Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment Regina Barzilay and Lillian Lee Department of Computer Science Cornell University Ithaca, NY 14853-7501 � regina,llee � @cs.cornell.edu Abstract We address the text-to-text generation prob- lem of sentence-level paraphrasing — a phe- nomenon distinct from and more difficult than word- or phrase-level paraphrasing. Our ap- proach applies multiple-sequence alignment to sentences gathered from unannotated compara- ble corpora: it learns a set of paraphrasing pat- terns represented by word lattice pairs and au- tomatically determines how to apply these pat- terns to rewrite new sentences. The results of our evaluation experiments show that the sys- tem derives accurate paraphrases, outperform- ing baseline systems. 1 Introduction This is a late parrot! It’s a stiff! Bereft of life, it rests in peace! If you hadn’t nailed him to the perch he would be pushing up the daisies! Its metabolical processes are of interest only to historians! It’s hopped the twig! It’s shuffled off this mortal coil! It’s rung down the curtain and joined the choir invisible! This is an EX- PARROT! — Monty Python, “Pet Shop” A mechanism for automatically generating multiple paraphrases of a given sentence would be of signif- icant practical import for text-to-text generation sys- tems. Applications include summarization (Knight and Marcu, 2000) and rewriting (Chandrasekar and Banga- lore, 1997): both could employ such a mechanism to produce candidate sentence paraphrases that other system components would filter for length, sophistication level, and so forth.1 Not surprisingly, therefore, paraphrasing has been a focus of generation research for quite some 1Another interesting application, somewhat tangential to generation, would be to expand existing corpora by providing time (McKeown, 1979; Meteer and Shaked, 1988; Dras, 1999). One might initially suppose that sentence-level para- phrasing is simply the result of word-for-word or phrase- by-phrase substitution applied in a domain- and context- independent fashion. However, in studies of para- phrases across several domains (Iordanskaja et al., 1991; Robin, 1994; McKeown et al., 1994), this was gen- erally not the case. For instance, consider the fol- lowing two sentences (similar to examples found in Smadja and McKeown (1991)): After the latest Fed rate cut, stocks rose across the board. Winners strongly outpaced losers after Greenspan cut in- terest rates again. Observe that “Fed” (Federal Reserve) and “Greenspan” are interchangeable only in the domain of US financial matters. Also, note that one cannot draw one-to-one cor- respondences between single words or phrases. For in- stance, nothing in the second sentence is really equiva- lent to “across the board”; we can only say that the en- tire clauses “stocks rose across the board” and “winners strongly outpaced losers” are paraphrases. This evidence suggests two consequences: (1) we cannot rely solely on generic domain-independentlexical resources for the task of paraphrasing, and (2) sentence-level paraphrasing is an important problem extending beyond that of paraphrasing smaller lexical units. Our work presents a novel knowledge-lean algorithm that uses multiple-sequence alignment (MSA) to learn to generate sentence-level paraphrases essentially from unannotated corpus data alone. In …

Content truncated to 500 words