News2vec: News Network Embedding with Subnode Information

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4843–4852, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4843 News2vec: News Network Embedding with Subnode Information Ye Ma1, Lu Zong1*, Yikang Yang2 and Jionglong Su1 1Mathematical Sciences, Xi’an Jiaotong-Liverpool University, Suzhou, 215028, China 2Mathematics and Statistics, Georgetwon University, Washington DC, 20007, US {ye.ma, lu.zong, jionglong.su}@xjtlu.edu.cn yy494@georgetown.edu Abstract With the development of NLP technologies, news can be automatically categorized and la- beled according to a variety of characteristics, at the same time be represented as low di- mensional embeddings. However, it lacks a systematic approach that effectively integrates the inherited features and inter-textual knowl- edge of news to represent the collective infor- mation with a dense vector. With the aim of filling this gap, the News2vec model is pro- posed to allow the distributed representation of news taking into account its associated fea- tures. To describe the cross-document linkages between news, a network consisting of news and its attributes is constructed. Moreover, the News2vec model treats the news node as a bag of features by developing the Subnode model. Based on the biased random walk and the skip-gram model, each news feature is mapped to a vector, and the news is thus represented as the sum of its features. This approach offers an easy solution to create em- beddings for unseen news nodes based on its attributes. To evaluate our model, dimension reduction plots and correlation heat-maps are created to visualize the news vectors, together with the application of two downstream tasks, the stock movement prediction and news rec- ommendation. By comparing with other es- tablished text/sentence embedding models, we show that News2vec achieves state-of-the-art performance on these news-related tasks. 1 Introduction News, carrying a large amount of information, can often guide public opinions, affect people’s behav- ior and drive the evolution of social events. In the era of information, news text is considered to be a part of big data that is continuously updated. In order to facilitate the performance of downstream NLP tasks, it is rather essential to find an efficient way to represent news as continuous vectors that contain the collective information of its inherited features and the inter-textual knowledge between different news. The most straightforward approach to embed news is to directly treat it as textual data and ap- ply text/sentence embedding models. To repre- sent texts/sentences as vectors, one of the classic techniques is to perform numerical operations on the word vectors (Mikolov et al., 2013b), which is recognized as a simple but powerful baseline (Conneau et al., 2018). SDAE (Vincent et al., 2010) uses an auto-encoder to compress texts into a low-dimensional vector. Paragraph Vector (Le and Mikolov, 2014) learns the distributed rep- resentation of sentences and documents by pre- dicting their contexts, i.e., words in the sen- tence/document. Additionally, Skip-Thought (Li and Hovy, 2014), FastSent (Hill et al., 2016) and Quick-Though (Logeswaran and Lee, 2018) learn distributed …

Content truncated to 500 words