Monolingual Machine Translation for Paraphrase Generation

Monolingual Machine Translation for Paraphrase Generation Chris QUIRK, Chris BROCKETT and William DOLAN Natural Language Processing Group Microsoft Research One Microsoft Way Redmond, WA 90852 USA {chrisq,chrisbkt,billdol}@microsoft.com Abstract We apply statistical machine translation (SMT) tools to generate novel paraphrases of input sentences in the same language. The system is trained on large volumes of sentence pairs automatically extracted from clustered news articles available on the World Wide Web. Alignment Error Rate (AER) is measured to gauge the quality of the resulting corpus. A monotone phrasal decoder generates contextual replacements. Human evaluation shows that this system outperforms baseline paraphrase generation techniques and, in a departure from previ- ous work, offers better coverage and scal- ability than the current best-of-breed paraphrasing approaches. 1 Introduction The ability to categorize distinct word sequences as “meaning the same thing” is vital to applications as diverse as search, summarization, dialog, and question answering. Recent research has treated paraphrase acquisition and generation as a machine learning problem (Barzilay & McKeown, 2001; Lin & Pantel, 2002; Shinyama et al, 2002, Barzilay & Lee, 2003, Pang et al., 2003). We approach this problem as one of statistical machine translation (SMT), within the noisy channel model of Brown et al. (1993). That is, we seek to identify the opti- mal paraphrase T* of a sentence S by finding: ( ) { } { }) )P( | maxP( arg | argmaxP * T T S TS T T T = = T and S being sentences in the same language. We describe and evaluate an SMT-based para- phrase generation system that utilizes a monotone phrasal decoder to generate meaning-preserving paraphrases across multiple domains. By adopting at the outset a paradigm geared toward generating sentences, this approach overcomes many prob- lems encountered by task-specific approaches. In particular, we show that SMT techniques can be extended to paraphrase given sufficient monolin- gual parallel data.1 We show that a huge corpus of comparable and alignable sentence pairs can be culled from ready-made topical/temporal clusters of news articles gathered on a daily basis from thousands of sources on the World Wide Web, thereby permitting the system to operate outside the narrow domains typical of existing systems. 2 Related work Until recently, efforts in paraphrase were not strongly focused on generation and relied primarily on narrow data sources. One data source has been multiple translations of classic literary works (Bar- zilay & McKeown 2001; Ibrahim 2002; Ibrahim et al. 2003). Pang et al. (2003) obtain parallel mono- lingual texts from a set of 100 multiply-translated news articles. While translation-based approaches to obtaining data do address the problem of how to identify two strings as meaning the same thing, they are limited in scalability owing to the diffi- culty (and expense) of obtaining large quantities of multiply-translated source documents. Other researchers have sought to identify pat- terns in large unannotated monolingual corpora. Lin & Pantel (2002) derive inference rules by pars- ing text fragments and extracting semantically similar paths. Shinyama et al. (2002) identify de- pendency paths …

Content truncated to 500 words