Using Dependency-Based Features to Take the ’Para-farce’ out of Paraphrase

Proceedings of the 2006 Australasian Language Technology Workshop (ALTW2006), pages 131–138. Using Dependency-Based Features to Take the “Para-farce” out of Paraphrase Stephen Wan†‡ Mark Dras† Robert Dale† †Center for Language Technology Div of Information Communication Sciences Macquarie University Sydney, NSW 2113 swan,madras,rdale@ics.mq.edu.au C´ecile Paris‡ ‡Information and Communication Technologies CSIRO Sydney, Australia Cecile.Paris@csiro.au Abstract As research in text-to-text paraphrase gen- eration progresses, it has the potential to improve the quality of generated text. However, the use of paraphrase genera- tion methods creates a secondary problem. We must ensure that generated novel sen- tences are not inconsistent with the text from which it was generated. We propose a machine learning approach be used to filter out inconsistent novel sentences, or False Paraphrases. To train such a filter, we use the Microsoft Research Paraphrase corpus and investigate whether features based on syntactic dependencies can aid us in this task. Like Finch et al. (2005), we obtain a classification accuracy of 75.6%, the best known performance for this cor- pus. We also examine the strengths and weaknesses of dependency based features and conclude that they may be useful in more accurately classifying cases of False Paraphrase. 1 Introduction In recent years, interest has grown in paraphrase generation methods. The use of paraphrase gen- eration tools has been envisaged for applications ranging from abstract-like summarisation (see for example, Barzilay and Lee (2003), Daum´e and Marcu (2005), Wan et al. (2005)), question- answering (for example, Marsi and Krahmer (2005)) and Machine Translation Evaluation (for example, Bannard and Callison-Burch (2005) and Yves Lepage (2005)). These approaches all em- ploy a loose definition of paraphrase attributable to Dras (1999), who defines a ‘paraphrase pair’ operationally to be “a pair of units of text deemed to be interchangeable”. Notably, such a definition of paraphrase lends itself easily to corpora based methods. Furthermore, what the more modern ap- proaches share is the fact that often they generate new paraphrases from raw text not semantic rep- resentations. The generation of paraphrases from raw text is a specific type of what is commonly referred to as text-to-text generation (Barzilay and Lee, 2003). As techniques for generating paraphrases im- prove and basic concerns such as grammaticality are less of an issue, we are faced with an addi- tional concern. That is, we must validate whether or not the generated novel sentence is in fact a paraphrase. It may be detrimental in some appli- cations, for example abstract-like summarisation, to allow a novel sentence that is inconsistent with the content of the input text to be presented to the end user. As an example of the type of inconsistencies that can arise from paraphrase generation, in Fig- ure 1, we present two examples of generated sen- tences. In each example, a sentence pair is pre- sented in which the second sentence was gener- ated from an input news article statistically us- ing a four-gram language model and a probabilis- tic word selection module. Although other para- phrase generation approaches differ in their un- derlying …

Content truncated to 500 words