BiSECT: Learning to Split and Rephrase Sentences with Bitexts

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6193–6209 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6193 BISECT: Learning to Split and Rephrase Sentences with Bitexts Joongwon Kim1∗, Mounica Maddela2∗, Reno Kriz,1,3 Wei Xu,2 Chris Callison-Burch1 1Department of Computer and Information Science, University of Pennsylvania, 2School of Interactive Computing, Georgia Institute of Technology 3 Human Language Technology Center of Excellence, Johns Hopkins University {jkim0118, ccb}@seas.upenn.edu, {mmadela3, wei.xu}@cc.gatech.edu, rkriz1@jh.edu Abstract An important task in NLP applications such as sentence simplification is the ability to take a long, complex sentence and split it into shorter sentences, rephrasing as necessary. We intro- duce a novel dataset and a new model for this ‘split and rephrase’ task. Our BISECT train- ing data consists of 1 million long English sen- tences paired with shorter, meaning-equivalent English sentences. We obtain these by extract- ing 1-2 sentence alignments in bilingual par- allel corpora and then using machine transla- tion to convert both sides of the corpus into the same language. BISECT contains higher quality training examples than previous Split and Rephrase corpora, with sentence splits that require more significant modifications. We cat- egorize examples in our corpus, and use these categories in a novel model that allows us to target specific regions of the input sentence to be split and edited. Moreover, we show that models trained on BISECT can perform a wider variety of split operations and improve upon previous state-of-the-art approaches in automatic and human evaluations.1 1 Introduction Understanding long and complex sentences is chal- lenging for both humans and NLP models. NLP tasks like machine translation (Pouget-Abadie et al., 2014; Koehn and Knowles, 2017) and dependency parsing (McDonald and Nivre, 2011) tend to per- form poorly on long sentences. Text simplification (Zhu et al., 2010; Xu et al., 2015) is often formu- lated with a specific step to break longer sentences into shorter sentences. This task is referred to as Split and Rephrase (Narayan et al., 2017). Several past efforts have created Split and Rephrase training sets, which consist of long, com- plex input sentences paired with multiple shorter ∗ Equal contribution. 1Our code and data are available at https://github. com/mounicam/BiSECT. Figure 1: The process of creating the English BISECT Split and Rephrase corpus. sentences that preserve the meaning of the input sentence. Narayan et al. (2017) introduced the WEBSPLIT corpus based on decomposing a long sentence into RDF triples (a form of semantic rep- resentation), and generating shorter sentences from subsets of these triples. However, the reliance on RDF triples and a limited vocabulary results in unnatural expressions (Botha et al., 2018) and re- peated syntactic patterns (Zhang et al., 2020a). More recently, the WIKISPLIT corpus (Botha et al., 2018) was introduced. It contains one mil- lion training examples of sentence splitting that were mined from the revision history of English Wikipedia. While this yields an impressive num- ber of training examples, the data are often quite noisy, with around 25% of WIKISPLIT pairs con- taining significant …

Content truncated to 500 words