Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5077–5089, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5077 Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation Ze Yang†, Can Xu♦, Wei Wu♦, Zhoujun Li†∗ †State Key Lab of Software Development Environment, Beihang University, Beijing, China ♦Microsoft Corporation, Beijing, China {tobey, lizj}@buaa.edu.cn {wuwei, caxu}@microsoft.com Abstract Automatic news comment generation is a new testbed for techniques of natural language gen- eration. In this paper, we propose a “read- attend-comment” procedure for news com- ment generation and formalize the procedure with a reading network and a generation net- work. The reading network comprehends a news article and distills some important points from it, then the generation network creates a comment by attending to the extracted discrete points and the news title. We optimize the model in an end-to-end manner by maximiz- ing a variational lower bound of the true objec- tive using the back-propagation algorithm. Ex- perimental results on two datasets indicate that our model can significantly outperform exist- ing methods in terms of both automatic evalu- ation and human judgment. 1 Introduction In this work, we study the problem of automatic news comment generation, which is a less ex- plored task in the literature of natural language generation (Gatt and Krahmer, 2018). We are aware that numerous uses of these techniques can pose ethical issues and that best practices will be necessary for guiding applications. In particular, we note that people expect comments on news to be made by people. Thus, there is a risk that peo- ple and organizations could use these techniques at scale to feign comments coming from people for purposes of political manipulation or persuasion. In our intended target use, we explicitly disclose the generated comments on news as being formu- lated automatically by an entertaining and engag- ing chatbot (Shum et al., 2018). Also, we under- stand that the behaviors of deployed systems may need to be monitored and guided with methods, including post-processing techniques (van Aken ∗ Corresponding Author Title: FIFA rankings: France number one, Croatia and England soar, Germany and Argentina plummet Body (truncated): World Cup glory has propelled France to the top of FIFA’s latest world rankings, with the impact of Russia 2018 felt for better or worse among a number of football’s heavyweight nations. These are the first set of rankings released under FIFA’s new formula that ”relies on adding/subtracting points won or lost for a game to/from the previous point totals rather than averaging game points over a given time period”. FIFA world rankings: 1.France 2.Belgium 3. Brazil 4. Croatia 5. Uruguay 6. England 7. Portugal 8. Switzer- land 9. Spain 10. Denmark Comment A: If it’s heavily based on the 2018 WC, hence England leaping up the rankings, how are Brazil at 3? Comment B: England above Spain, Portugal and Ger- many. Interesting. Table 1: A news example from …

Content truncated to 500 words