Neural Network Alignment for Sentential Paraphrases

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4724–4735 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4724 Neural Network Alignment for Sentential Paraphrases Jessica Ouyang and Kathleen McKeown Deptartment of Computer Science Columbia University New York, NY 10027 {ouyangj, kathy}@cs.columbia.edu Abstract We present a monolingual alignment system for long, sentence- or clause-level alignments, and demonstrate that systems designed for word- or short phrase-based alignment are ill- suited for these longer alignments. Our sys- tem is capable of aligning semantically simi- lar spans of arbitrary length. We achieve sig- nificantly higher recall on aligning phrases of four or more words and outperform state-of- the-art aligners on the long alignments in the MSR RTE corpus. 1 Introduction Monolingual paraphrase alignment is an active area of research, with applications in many natu- ral language processing tasks, such as text-to-text generation (Barzilay and Elhadad, 2003; Barzi- lay and McKeown, 2005), natural language infer- ence (MacCartney et al., 2008), and recognizing textual similarity (Sultan et al., 2014b). Madnani and Dorr (2010) identify three levels of paraphras- ing. The first is lexical paraphrasing, where in- dividual words are replaced by synonyms or hy- pernyms. The second, phrasal paraphrasing, in- volves equivalent idiomatic phrases, such as verb- preposition combinations (eg. “take over” or “as- sume control of”), or syntactic transformations, such as active versus passive voice. In this work, we focus on the third: sentential paraphrasing. Sentential paraphrasing can trivially be achieved by performing lexical and phrasal paraphrasing on parts of a sentence, but Madnani and Dorr note that more interesting paraphrases, such as “He needed to make a quick decision in that situation” and “The scenario required him to make a split-second judgment,” are challenging. Past work has focused on lexical and short phrasal alignments, in part because most exist- ing corpora consist of mostly word-level align- ments. Yao et al. (2013b) report that 95% of align- ments in the MSR RTE (Brockett, 2007) and Ed- inburgh++ (Cohn et al., 2008) corpora are single- token, lexical paraphrases, and phrases of four or more words are less than 1% of MSR RTE and 3% of Edinburgh++. In this work, we present a monolingual aligner for long phrasal and sentential paraphrases. Our contributions are as follows: • Our pointer-network-based system aligns phrases of arbitrary length. • Our system aligns directly at the phrase level by composing the semantics of the words in each phrase into a single representation of the meaning of the entire phrase. • We conduct experiments on aligning long paraphrases using the summarization corpus of Ouyang et al. (2017), the first use of this corpus for the alignment task, as well as the MSR RTE corpus (Brockett, 2007). • We achieve significant increases in recall (over 75 points) while also maintaining a strong lead in F-measure on aligning long paraphrases (involving phrases of four or more words), compared with existing state- of-the-art word- and phrase-based aligners. 2 Related Work The development of monolingual alignment …

Content truncated to 500 words