Inference Time Style Control for Summarization

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5942–5953 June 6–11, 2021. ©2021 Association for Computational Linguistics 5942 Inference Time Style Control for Summarization Shuyang Cao and Lu Wang Computer Science and Engineering University of Michigan Ann Arbor, MI {caoshuy, wangluxy}@umich.edu Abstract How to generate summaries of different styles without requiring corpora in the target styles, or training separate models? We present two novel methods that can be deployed dur- ing summary decoding on any pre-trained Transformer-based summarization model. (1) Decoder state adjustment instantly modifies decoder final states with externally trained style scorers, to iteratively refine the output against a target style. (2) Word unit prediction constrains the word usage to impose strong lex- ical control during generation. In experiments of summarizing with simplicity control, auto- matic evaluation and human judges both find our models producing outputs in simpler lan- guages while still informative. We also gen- erate news headlines with various ideological leanings, which can be distinguished by hu- mans with a reasonable probability. 1 Introduction Generating summaries with different language styles can benefit readers of varying literacy lev- els (Chandrasekaran et al., 2020) or interests (Jin et al., 2020). Significant progress has been made in abstractive summarization with large pre-trained Transformers (Dong et al., 2019; Lewis et al., 2020; Zhang et al., 2019; Raffel et al., 2019; Song et al., 2019). However, style-controlled summarization is much less studied (Chandrasekaran et al., 2020), and two key challenges have been identified: (1) lack of parallel data, and (2) expensive (re)training, e.g., separate summarizers must be trained or fine- tuned for a pre-defined set of styles (Zhang et al., 2018). Both challenges call for inference time methods built upon trained summarization mod- els, to adjust styles flexibly and efficiently. To address these challenges, we investigate just- in-time style control techniques that can be directly applied to any pre-trained sequence-to-sequence (seq2seq) summarization model. We study two methods that leverage external classifiers to favor Daily Mail Article: . . . ��� A 16-year-old who was born a girl but identifies as a boy has been granted the opportunity to go through male puberty thanks to hormone treatment. ��� . . . ��� The transgender boy, who has felt as though he is living in the wrong body since he was a child, has been given permission by a Brisbane-based judge to receive testosterone injections ��� . . . (a) Decoder State Adjustment: ��� Queensland teen has been granted hormone treatment. The 16-year-old was born a girl but identifies as a boy. ��� . . . ��� A judge has granted the teen permission to receive testosterone injections. ��� . . . (b) Word Unit Prediction: A 16-year-old who was born a girl has been given the right to go through male puberty. The transgender boy has lived in a female body since he was a . . . Figure 1: Sample summaries generated by our style control methods via (a) adjusting …

Content truncated to 500 words