Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1267 - 1280 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning Demian Gholipour Ghalandari1,2, Chris Hokamp1, Georgiana Ifrim2 1Aylien Ltd., Dublin, Ireland 2Insight Centre for Data Analytics, University College Dublin, Ireland 1{first-name}@aylien.com georgiana.ifrim@ucd.ie Abstract Sentence compression reduces the length of text by removing non-essential content while preserving important facts and grammaticality. Unsupervised objective driven methods for sentence compression can be used to create customized models without the need for ground-truth training data, while al- lowing flexibility in the objective function(s) that are used for learning and inference. Recent unsu- pervised sentence compression approaches use cus- tom objectives to guide discrete search; however, guided search is expensive at inference time. In this work, we explore the use of reinforcement learning to train effective sentence compression models that are also fast when generating predictions. In partic- ular, we cast the task as binary sequence labelling and fine-tune a pre-trained transformer using a sim- ple policy gradient approach. Our approach outper- forms other unsupervised models while also being more efficient at inference time. 1 Introduction In general, the information content of text is cor- related with its length. However, for a given text, a shorter version may still convey the essential in- formation while preserving grammaticality (Sid- dharthan, 2014). The definition of essential can change depending on the downstream application, thus models for text compression must be able to adapt based on information about the downstream task. Sentence compression models have been used as sub-modules of text and speech summarization (Banerjee et al., 2015; Shang et al., 2018), for head- line generation (Dorr et al., 2003), subtitle gener- ation (Vandeghinste and Pan, 2004), and summa- rizing emails (Zajic et al., 2008). Potential applica- tions also include snippet generation and highlight- ing for social media, blog posts or search results. Figure 1: Reinforcement learning framework for sen- tence compression. Given a particular text compression task, rele- vant evaluation metrics and auxiliary models of compression quality may not be straightforward to formulate as well-behaved differentiable objectives that can be used with standard backpropagation. In addition, ground-truth examples may be difficult to obtain because the annotation task is difficult to fully specify, and metrics which capture differ- ent facets of compression quality, such as fluency and optimal sentence length, may be negatively correlated. Even in the case where ground-truth examples are available, they are likely to represent only a subset of the possible outputs, so there is a risk of over-fitting or biasing models when relying solely upon a small amount of gold training data for optimization. Recent unsupervised sentence compression ap- proaches leverage powerful neural language mod- els to directly optimize objectives such as flu- ency and faithfulness of compressed sentences, us- ing discrete search strategies, without relying on ground-truth examples (Niu et al., 2019; Zhou and Rush, 2019; Schumann et al., 2020). However, these search-based methods are …

Content truncated to 500 words