Neural News Recommendation with Multi-Head Self-Attention

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6389–6394, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6389 Neural News Recommendation with Multi-Head Self-Attention Chuhan Wu1, Fangzhao Wu2, Suyu Ge1, Tao Qi1, Yongfeng Huang1,and Xing Xie2 1Department of Electronic Engineering, Tsinghua University, Beijing 100084, China 2Microsoft Research Asia, Beijing 100080, China {wu-ch19, gsy17, qit16, yfhuang}@mails.tsinghua.edu.cn, {fangzwu, xing.xie}@microsoft.com Abstract News recommendation can help users find in- terested news and alleviate information over- load. Precisely modeling news and users is critical for news recommendation, and cap- turing the contexts of words and news is im- portant to learn news and user representations. In this paper, we propose a neural news rec- ommendation approach with multi-head self- attention (NRMS). The core of our approach is a news encoder and a user encoder. In the news encoder, we use multi-head self-attentions to learn news representations from news titles by modeling the interactions between words. In the user encoder, we learn representations of users from their browsed news and use multi- head self-attention to capture the relatedness between the news. Besides, we apply addi- tive attention to learn more informative news and user representations by selecting impor- tant words and news. Experiments on a real- world dataset validate the effectiveness and ef- ficiency of our approach. 1 Introduction Online news platforms such as Google News1 and MSN News2 have attracted many users to read news online (Das et al., 2007). Massive news ar- ticles are generated everyday and it is impossible for users to read all news to find their interested content (Phelan et al., 2011). Thus, personalized news recommendation is very important for online news platforms to target user interests and allevi- ate information overload (IJntema et al., 2010). Learning accurate representations of news and users are two core tasks in news recommenda- tion (Okura et al., 2017). Several deep learn- ing based methods have been proposed for these tasks (?Kumar et al., 2017; Khattar et al., 2018; 1https://news.google.com/ 2https://www.msn.com/en-us/news Rockets trade Carter- Williams to the Bulls Best NBA Moments in 2018 James Harden's incredible heroics lift Rockets over Warriors in overtime Weather forecast This Week Figure 1: Several news browsed by an example user. Orange and green dashed lines represent the interac- tions between words and news respectively. Wu et al., 2019b,c,a; Zhu et al., 2019; An et al., 2019). For example, Okura et al. (2017) proposed to learn news representations from news bodies via auto-encoders, and learn representations of users from their browsed news via GRU. However, GRU is quite time-consuming, and their method cannot capture the contexts of words. Wang et al. (2018) proposed to learn news representations from news titles via a knowledge-aware convolutional neural network (CNN), and learn representations of users based on the similarities between candidate news and their browsed news. However, CNN can- not capture the long-distance contexts of words, and their method cannot model the relatedness be- tween …

Content truncated to 500 words