Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3065–3075, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3065 Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning Peng Xu, Chien-Sheng Wu, Andrea Madotto and Pascale Fung Center for Artificial Intelligence Research (CAiRE) Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology, Clear Water Bay [pxuab,cwuak,amadotto]@connect.ust.hk, pascale@ece.ust.hk Abstract Sensational headlines are headlines that cap- ture people’s attention and generate reader in- terest. Conventional abstractive headline gen- eration methods, unlike human writers, do not optimize for maximal reader attention. In this paper, we propose a model that generates sen- sational headlines without labeled data. We first train a sensationalism scorer by classi- fying online headlines with many comments (“clickbait”) against a baseline of headlines generated from a summarization model. The score from the sensationalism scorer is used as the reward for a reinforcement learner. How- ever, maximizing the noisy sensationalism re- ward will generate unnatural phrases instead of sensational headlines. To effectively lever- age this noisy reward, we propose a novel loss function, Auto-tuned Reinforcement Learning (ARL), to dynamically balance reinforcement learning (RL) with maximum likelihood esti- mation (MLE). Human evaluation shows that 60.8% of samples generated by our model are sensational, which is significantly better than the Pointer-Gen baseline (See et al., 2017) and other RL models. 1 Introduction Headline generation is the process of creating a headline-style sentence given an input article. The research community has been regarding the task of headline generation as a summarization task (Shen et al., 2017a), ignoring the fundamental dif- ferences between headlines and summaries. While summaries aim to contain most of the important information from the articles, headlines do not necessarily need to. Instead, a good headline needs to capture people’s attention and serve as an irresistible invitation for users to read through the article. For example, the headline “$2 Bil- lion Worth of Free Media for Trump”, which gives only an intriguing hint, is considered bet- ter than the summarization style headline “Mea- suring Trump’s Media Dominance” 1, as the for- mer gets almost three times the readers as the lat- ter. Generating headlines with many clicks is espe- cially important in this digital age, because many of the revenues of journalism come from online advertisements and getting more user clicks means being more competitive in the market. However, most existing websites 2 naively generate sensa- tional headlines using only keywords or templates. Instead, this paper aims to learn a model that gen- erates sensational headlines based on an input ar- ticle without labeled data. To generate sensational headlines, there are two main challenges. Firstly, there is a lack of sen- sationalism scorer to measure how sensational a headline is. Some researchers have tried to man- ually label headlines as clickbait or non-clickbait (Chakraborty et al., 2016; Potthast et al., 2018). However, these human-annotated datasets are usu- ally small and expensive …

Content truncated to 500 words