Unsupervised Rewriter for Multi-Sentence Compression

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2235–2240 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2235 Unsupervised Rewriter for Multi-Sentence Compression Yang Zhao1, Xiaoyu Shen2, Wei Bi3, Akiko Aizawa4,1 1The University of Tokyo, Tokyo, Japan zhao@is.s.u-tokyo.ac.jp 2Max Planck Institute for Informatics, Saarland, Germany xshen@mpi-inf.mpg.de 3Tencent AI Lab, Shenzhen, China victoriabi@tencent.com 4National Institute of Informatics, Tokyo, Japan aizawa@nii.ac.jp Abstract Multi-sentence compression (MSC) aims to generate a grammatical but reduced compres- sion from multiple input sentences while re- taining their key information. Previous dom- inating approach for MSC is the extraction- based word graph approach. A few vari- ants further leveraged lexical substitution to yield more abstractive compression. How- ever, two limitations exist. First, the word graph approach that simply concatenates frag- ments from multiple sentences may yield non- fluent or ungrammatical compression. Sec- ond, lexical substitution is often inappropri- ate without the consideration of context in- formation. To tackle the above-mentioned is- sues, we present a neural rewriter for multi- sentence compression that does not need any parallel corpus. Empirical studies have shown that our approach achieves comparable results upon automatic evaluation and improves the grammaticality of compression based on hu- man evaluation. A parallel corpus with more than 140,000 (sentence group, compression) pairs is also constructed as a by-product for fu- ture research. 1 Introduction Multi-sentence compression (MSC) aims to gener- ate a single shorter and grammatical sentence that preserves important information from a group of related sentences. Over the past decade, multi- sentence compression has attracted considerable attention owing to its potential applications, such as compressing the content to be displayed on screens with limited size (e.g., mobile devices) and benefiting other natural language processing tasks, such as multi-document summarization (Banerjee et al., 2015), opinion summarization, and text sim- plification. Most existing works rely on the word graph approach initialized in (Filippova, 2010), which offers a simple solution that copies frag- ments from different input sentences and concate- nates them to form the final compression. Later on, a bunch of subsequent research works (Boudin and Morin, 2013; Banerjee et al., 2015; Luong et al., 2015; ShafieiBavani et al., 2016; Pontes et al., 2018; Nayeem et al., 2018) attempted to im- prove the word graph approach using a variety of strategies, such as keyphrase re-ranking. However, such extraction-based approach may yield non- fluent or ungrammatical compression. A previous study (Nayeem and Chali, 2017) has shown that word graph approaches produce more than 30% of the ungrammatical sentences (evaluated by a chart parser), which is partly due to the non-usage of rewording by these extraction-based approaches. In fact, human annotators tend to compress a sen- tence through several rewriting operations, such as substitution and rewording (Cohn and Lapata, 2008). Despite some research works that attempt to do the lexical substitution, it is often inappro- priate without the consideration of context infor- mation. To tackle the above-mentioned problems, we present herein an unsupervised rewriter to improve the grammaticality of compression …

Content truncated to 500 words