Controllable Summarization with Constrained Markov Decision Process

Controllable Summarization with Constrained Markov Decision Process Hou Pong Chan1, Lu Wang2, and Irwin King3 1University of Macau, Macau SAR, China 2University of Michigan, Ann Arbor, MI, USA 3The Chinese University of Hong Kong, Hong Kong SAR, China 1hpchan@um.edu.mo 2wangluxy@umich.edu 3king@cse.cuhk.edu.hk Abstract We study controllable text summarization, which allows users to gain control on a particu- lar attribute (e.g., length limit) of the generated summaries. In this work, we propose a novel training framework based on Constrained Markov Decision Process (CMDP), which conveniently includes a reward function along with a set of constraints, to facilitate better summarization control. The reward function encourages the generation to resemble the human-written reference, while the constraints are used to explicitly prevent the generated summaries from violating user-imposed re- quirements. Our framework can be applied to control important attributes of summariza- tion, including length, covered entities, and abstractiveness, as we devise specific con- straints for each of these aspects. Extensive experiments on popular benchmarks show that our CMDP framework helps generate informa- tive summaries while complying with a given attribute’s requirement.1 1 Introduction Text summarization aims to condense the informa- tion of an input document into a concise summary. Although recently neural abstractive summariza- tion models have achieved promising performance (See et al., 2017; Paulus et al., 2018), they do not allow users to indicate their preference to control different aspects of the generated summaries. Con- trollable summarization has many use cases. For instance, it can summarize product descriptions to fit within a word limit in online advertising. In another example, teachers can demonstrate the technique of paraphrasing important information by showing a system-generated summary with 1Our source code is available at https://github .com/kenchan0226/control-sum-cmdp. high abstractiveness. Controllable summarization can also complement information retrieval sys- tems, for example, to only generate summaries covering the entities that users are interested in. Figure 1 illustrates one such usage, where our proposed model produces distinct abstractive sum- maries of the same source document, focusing on different input entities. To allow users to control a particular attribute of the generated summaries, Fan et al. (2018) proposed a token-based controllable summariza- tion model (ControlSum). Although ControlSum incorporates control tokens that let users specify a requirement on a summary attribute, the max- imum likelihood training objective of the model does not provide explicit supervision signals that prevent the model from violating the specified attribute requirement. Consequently, a substantial portion of the generated summaries still fail to meet the specified attribute requirement as shown in our experiments. One possible solution to enforce the attribute re- quirement is to apply reinforcement learning (RL) with Markov Decision Process (MDP) (Bellman, 1957) to optimize a weighted sum of reward func- tions, including a penalty function to penalize the violation of the attribute requirement, and a summarization metric to encourage the generated summaries to be consistent with the references. However, selecting appropriate weights for dif- ferent reward functions is a delicate task, and requires intensive hyperparameter tuning. In this work, we argue that applying constraints on the …

Content truncated to 500 words